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.This project had three major goals: (l)__To develop a 
handbook for high school teachers , that shows how instructioh and 
testing can work together to foster student learning? (2) To pilot 
test this handbook'in .the context of a staf T devfelopment program for 
high school teachers; and (3j To prepare training manuals that would 
allow the staff development program to be . implemented in school 
districts that had not participated in its development or. pilot 
testing. The handbook • sets forth a goal-based approaph to integratifig 
teaching and testing which intends to reflect the contextual factors 
that inflijpnci teacher practidejs, e.g., student characteristics. The 
.first section of this repojrt presents an overview of the ^processes 
followed in developing the handhook, the issues«.f aced in determining - 
its design, and the distinctive characteristics of the final product. 
The second section describes the .development and pilot-testing of a 
staff development program based on the 'handbook . In the final ' 
section, descriptions are- of fered of the content and organization of 
thre« manuals developed byithe project: the "Resource Guide for 
Teachers the "Planning Guide for Lead Teachers and Principals," and 
the "Program Gu^de for Superintendents." This jsection also contains a 
description ' of the refinement process and field trials that are ' 
planned for the manuals. Included in the appendixes are a review of 
the literature on .integrating teaching and testing, and the 
questionnaire used to gather information about high school teachers' 
testing procedures. (JD) ^ 
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G0AI4 OF THE PROJECT 

This project had tjjree major goals: ' \ ' \' ' . * 

. (i) To develop* handbook for high sehool teaehersXthat phows how instrueflin 

\ • \ ' 

^ and testing c|ji work together to foster student learning; , 
m To,pilot test this hw'dbook in the context pf a.staff development' program 
for high school teachers; and 
, (3) Tq prepare training manuals that would allow the..staff development, program 
tp. be Implemented in school districts that had hot participated in its 
development or pilot testing! 
The development of a handbook on Hnking instruction and assessment seemed of 
crucial impprtance both because of the content wiih^hich it was to deal and the form 
it was to take From the standpoint of content; the idea of a handbook (appeared timely 
in view of moun«n| evidence that the quaUty of teaching and tSing i^^high Lhool 
courses is far from satisfactory. ^S^dies of high school classrooms and teachers iuggest 
the following conclusions. ' . ^ . • . ' ^ / ' ' 

1. Instruction across aU subject areas generally is geared to the lSWest levels 
of learning and consists largely of lectures. ,foUowed by question-an^jsrer 

^ sessions. ^ Textbooks typically define the seppe and sequence of' instruction*. 

V 

Classroom assessment by ahd large consists of quizzes and objective tests 

9 

that call for factua;. responses (Sirotnik. 1982). ' 

J , • • * 

2. Teachers <Jn the whole -are poorly irained in the area o? assessment, and 

high school teachers receive even less training than ' elementary School 
teachers (Stiggins & Bridj^eford,. 1^82). To the extent that stkff development 



{ 



programs this area are available, they generaUy focus oh administering 
and interpreting tests mandated and managed by ^either the school district 
or the state (Bucry et al.. 1982). 



3. Without specific tpainfng, teachers are, apt to have difficulty ijn' designihg 

\ assessment tools that are tied closely to inSstructionBl g\?^is. .Indeed, a r^cent^ 

• X- ^ * ■^ * ' ' • • . 

^ study suggests that secondary school teachers may have difficulty\in 

determining whether a given set-of tesr items corresponds or fails to 

correst^ond to a given learning. goal' (Carter, 1984). 

\ ' It appears that classroom tests are used primarily for purposes of grading. 

Teachers do not regularly use tests-as a gaide to instructional planning and 

decision-miking or ;as an aid to student, learq^lng (Rudmanret al„ 1988) 

•Students receive much less fe.edback on. their learning progress than the 

, .. literature on effective instruction in general and mastery learning in particular 

> suggests' is desirable (Sirotnik, 1983).' 

. The proposed content of the handbook thus appeared to respond to a lai*Be-scale 

problem in our nation's secondary sch^ls. The intended tocus ot the 'handb&, on 

linking instruction and assessm^ent, seemed particularly appropriate, moreover, in .view 

of ,the fact that most texts for teachers or prospective teachers have focused on either 

instructiort or assessment, rather than on the connections-between the %vio.. For example. 

Gagne' and Briggs'. Prim/iples oMn^ructional Design (Holt, Rinehart. arid Winston. 

1979), a highly- regarded afid widely used text in schools*bf teacher education, treats 

issues of learning anti teaching extensively, but only U chapters 'out of 15 in tha text 

deal directly with the assessment of student learning. In a similar manner. Brown's 

Measuring- Classroom Achievement (Holt, Rinehart and Winston, 1981), a well-respfected 

text, deals thoroughly with' topics iri measurement, ^but contains oniyone QhapteV out 

of ten on the use of tests in the instructional process. The anticip.ated- emphasis of 

the handbT)ok on integrating teaching and testing Ihus promised to fill a need that 

\ ' ^ . 

rarely was addressed full^ in available texts. • 



The proposed form of the handbook also 'seemed disJtincUve, Most texts that 
deal v/ith cllssroom instruction or assessment are too long and detailed for practical 
use by high school teachers. Even those feiw texts that do give substantial attention 
to both teaching and testing, for 'example, L earning and Human; Abilities, by Klausmeir 
(Harper and Row, 1975) and Evaluation to Improve Learning^ by' Bloom, Madaus, artd^ 
Hastings (McGraw Hijl, ,1981), are generally intented to serve as comprehensive texts 
appropriate for in-depth study. A handbooK that would be substantive, but at the same 
time relatively, easy to use in the conteart of inservice courses and programs for practicing 
teachers, seemed to be needed. ' • * 

The development o^ an inservice prograjn and the preparation of Related training 

manuals to support the Handbook was viewed as essential in light of all that is known 

... •/ ♦ • - . 

about the . conditions that f ostep the adoption of innovations in -schools^ It is commonly 

observed that handbooks, texts, articles, and similar materials^ no matter how well- 

conceived or written, rarely in themselves have much impact on teacher practice. 

' ' . ■■ ' ■ . * ' 

Programs must*be created that provide teachers with the time, resourses, and assistance 

'» . .. » " • ■ 

needed to /translate new ideas and procedures into practice. "Research on staff 
development, as well as our own experience"* in .school improvement efforts, suggest, 
furthermore, that a variety of roles and responsibilities needs to be carried out in a 
^ school district to fulljj implement an inservice prof^ro of the kind pilot tested through 
^ this project. It."appeared crucial to prepare guides for superintendents, principals and 
department chairs (oc "lead teachers"), as well as participating teachers*, that would 
clarify the tasks each group was to perform. 
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' ^ DEVELOPMENT OF TH5 HANDBOOk • 

» • ■ • 

♦ • 
Overview * 

From the outset, the H andbook was Int^oted to bei>oth cf -ceptually sophisticated 
• and, practical. We had a commitment/ to draw upon^ the i .liable ideas and 
^ procedves for connecting instruction and assessment,' feod to synthesis**, and present 
this material in a forih that high school teachers would >fihd meaninefuL 

To help«)assure that the handbook would be both substai^t!'vsl:^ sound and practical, 
three different advisory panels j#?ere formed in the^fall of 1982, soon after '.he contract 
was ay^rded: (1) a technicaiySnel on mi^asurement; (2) a technical panel on instruction; 
/and (3) a panel q£ practicing high SQhbol teachers. Members of these panels are' listed, 
in Appendix 1. In addition. Dr. Jason MiUman, an eminent authority in testing and the 
''use of 'test information, was asked to serve as a consultant to the project and, in this 
capacity, tp help integrate recommendations from all three panels about the design a^hd 
'development of the Handbook* ^ , - i. 

Under' the guidance of the advisory panels and Dr. MiUman;** the Handbook 

■ N 

n progressed through five phases of development: " ' . 

(1) Conceptualization and design (fall 1982 • winter 1983); 
. (2) Initial draft of selected chapters (spring 19 83^ 

(3) Revision, and preparation of a complete draft (summer-fall 1983); 
^ (4) Pilot testing and subsequent revision (spring -^summer 1984); and 
•(5) Final review and revision'' (fall 1984 - January L9C5) ' 

Issues Faced in Designing and Drafting the Handbook 

' - • • > y 

The most basic and difficult issue faced in designing the handbook concerned its 

'» 

' underlying structure. We wanted to find a theme or set of themes around which 

» • . ■ 

' ^Members oX the teacher panel met ' five times -^between fall of 1982 and fall of 1983. 
Members of the technical panels did not meet 'as a group, but rather provided^ ideas 
and suggestions by mail and phone. Particularly extensive guidance was furnished by 
the chairs of the technical panels, DrS. Ronald Hambletpn and M. David Merrill, a$ well 
• as Dr. Millman. 

o • • , • 6 : • • 
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aiscussions of teaching and testing could be organized. Tliis^' proved to be a more 
dejuanding task than originally anticipated.- A number of competing frameworks for 
organizing the- handbook were 9onsidered. 

One set of -organizers ^discussed was based on tjie different purposes served by 
assessment, e.g.. to ^plan instruction in the beginning of a course, to assess students* 
learning progress, to verify students' mastery over a learning area, etc. If tljis framework 

•were adopted, the Handbook would focus on designing, selecting, or adapting teste to 

. • ' ♦ 

respond to each of the -major types of information needs that teachers have. 

Another wganlzational structure Hhat was dis&ssed centered on instructional 

models and (nethods. If this structure we/e chosen, ^e handbook would focus on the 

implications for instructional planning and assessment of different approaches to 

instructloix, either, general approaches, li^fe mastery learning, or more specific methods, 

like.simjilatlon, lectured or disc^slon. Th§ use of Infotmal methods of assessment, like 

asking questions in class. Vr Observing students during small group work would ^eed to 

be described in considerable detail if this framework were adopted. 

.A third proposal for organizing the Handbook was to focus on the influence that 

' - •. . . 
^lifferent types of learning goals have on instruction and as^essm^nt. What and how 

well students are expected to learn determines in large part) how instruction and 

assessment are designed. Were this framework adopted, the handbook would include 

discussions of the kinds and levels of learning outcomes that- might be expected from 

instruction in Jiigh school, and guidelines for matching instruction aQd assessment to 

these goals. 

Each of these candidates for an underlying- structure*. appeared to have merit.. 
No single framework in itself seemed adequate. Many members of both the Teachers' 
Advisory Panel and of the T^hnical Panels, for example, made clear that none of these 
frameworks was likely to permit sufficient discussion of principles for writing technically- 
sound test items and other assessment procedures. Teachers * receiv^" such limited 



preparation in this area, that many of our. advisors held the view that item writing 

procedures needed to be addressed in some detail. 

' '. . ■ ... . ^ 

{f ' • 
of discussions about alternative frameworks, we completed a r^yiew 

. , . ^ \- ' 

of the literature on integrating teaching and testing and a sii^ey of high school teacher's ' 
testing practices. The lifferature' review is presented in Appendix 2. Thsi. survey 
instrument Is presented 'and diSQUssed in- Appendix 3. These activities provided sotne 

.■ ♦ ' * 

assistance in establishing a focus for the -handbook. ^We found through the 3urvey, for 

example, that telichers .expressed .much more Interest in learning "how to design tests 

that are matched' closely to what 1 have taught" than other proposed topics Qoncerning. 

the . teaching-testing connect^.on, like "how to use test results as a guide for planning 

rem^ial or. enrichment activities," or "how to score and manage test information with 

the aid) of a micro-computer." > ^ • 

After extensive discussion withNhe Teachers' Advisory^ Panel, and the preparation ' 

^nd review by members of both the teaqher and experts* panels of several draft chapters. 

^ , of the.; >Handbook. it was decided that the primary conceptual focus for the* handbook 

would be the idea of .anchoring. leaetiing, testing, and the use of test information ,to.^ 

the learning oi^tcopfies expected from instruction, but that the main elements of , the* . 

other frameworks referred to above klso w^uld be introWed. The decision to focus 

on-learning goals as a cornerstone for teaching and testing stemmed largely from the 

perception that this would permit us to deal more fully with the content of instruction . 

and assessment than would frameworks based on instructional models or the purposes 

served by tests. This seemed particularly important at the high school level, where so 

much of what teachers do depends on the kind of content with which they are dealing. 

Once the focusi of the Hanc'^ook was clarified, attention turned to establishi^ng 

an organizatlbnal structure for the Handbook. The most basic organizational decision 

that was made involved dividing the Handbook into two broad sections. The, first, to , 

be Called "Foundations,*^ was to introduce the overall idea of integrating teaching and 

"» 

./ . • . ... * 
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testing and discuss the various aspects^of learning, tcaclj^ end testing that need to 
be coordinated to achieve this integration. The foundatiois section would also indicate 
to the reader that a go4l.based approach to linking . instruction and assesfeiuent iwouid 
be emphasized* In the Handbook, , ' 

sl^he second section 0/ the book, it was further d^ed. would deal With 
Applications." This would provide guideUncs for using the ideas addressed in the 
foundation? section in a deliberate and well-sequeneed fashion. . Chapters were' to deal 
with forraulatin:^ learnings goals tor students, matching instructiSn and assessment to 
these goals.^uring that tests meet standards of technical adequacy and are appropriate 
for one's particular information needs, and so forth. The organizational pattern agreed 
to. is indicated by the table of Contents of the Handbook. - which is presented on^'the 

After Agreeing on an organizational structure, a couple of months were spent in^ 
estabUshing '^n approach to classifying- difterent types 4nd levels of learning goals that . 
would serve as an effective guide for instruction, assessment, and the inUrpretatidn of 
assessment results.^ The mi^ ob^^ous approach to use was Bloom's well-known taxonomy 
for classifying goak in the cognitive domain. However, after discussion with the 
Teachers' Panel and In light of our previous experience in applying the taxonomy, we 
decided that the framework was not satisfactory for our purposes. One reason for t s 
decision was that it was 'difficult for our teacher advisors, most of whom who had 
.Studied the taxonomy before, to ' differentiate consistently . among the six levels of 
learning described in the taxonomy. Furthermore, the taxonomy was silent about the 
connections between knowledge (level 1 of the taxonomy) and the general intellectual 
skills. represented In levels 2 through 6. Finally, concern was e^fpressed about the 
assumption underlying the taxonomy that learning at one level necessarily depended on 
learning at- a lower level. Did "evaluation" invariably . Involve a hi^er level of 
performaJice than "synthesis"? Bloom's taxonomy, while undoubtedly a major contribution 
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INTEGRATING TEACHINQ AND TESTINQ: 
A. Handbook for High School Tei^ehers 



Intvoduetion 



PART I: FOUNDATIONS , 

Chapter 1: What does It me&n to integrate teaching and testing? 
Chapter 2: Dimensions of learning that iAfluence te'aeliing and te^ng 
Chapter 3: Dimensions-of context that influence teaching end tesi^g 

Chapter 4: Purposes served by tests 

Chiiipter 5: Types of test items- and assessment procedures 



PART II: APPPCATIONS 

Chapter 6: Formulating expected learning outcomes ^ 



"Chapter 7: Matching teaching to expected learning outcomes ^. 

Chapter 8: Matching test items and assessment procedures ' 
to expected, learning outcomes 



Chapter 9: Assuring quality in tests items apd assessment procedures . 

Chapttf 10: Preparing, administering and scoring tests 

Chapter 11: Using test information as feedback to students , 

Chapter. 12: Using test in^ormatio;i-as a guide to instructional decisions 
Chapter IS: Using test jinformation in grading 
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• , • • 

to eOucational practice, did not seem entirely appr6priate for use in the HantJbook. 



Various other perspectives on learning g9als were studied and discussed. Including 
those developed by Gagne, by Posner and Ru^nitsky, and by Merrill. Finally, we decided 
to adapt Merrill's framework as ^n anchor for th e Handbook, largely because this brought 
into focus the connections between knowledge and skill, and because it was relatively 
easy to differentiate among the three levels ^of accomplishment described In the 
framework (although we described these, levels in different tewas-thin' Merrill). More 
information on our choice of a learning taxonomy \t -contained in Chapter Two of the 
Handbook. 

Two other content-related issues demanded considerable attention. Cue had to 
do with our stance toward informal assessment, for example, the questions teachers ask 
in class to check students' understanding during a lesson and the unstructured or 
"spontaneous** observations teachers > make of students as they work on individual or 
group assignments. It was acknowledged thL"- '.iformal assessment is essential to managing 
instruction and providing feedback to stur ^, but. we vdecided t lat we would give 
priority to more formal assessment, that is, assessment based on deliberately structured 
tasks and supported by well-defined rules for evaluating students* responses. This 
decision seemed appropriate from the teacher advisors* vantage point in view of the 
wide experience that teachers have in informal assessment and the fact that numerous 
inservice programs already have been developed that give considerable attention to this 
topic. ' # ' . . 

Another content-related issue that arose concerned the treatment to be given to 
re ent technical advances in domain-specification strategies, standards-setting 
procedures, item analysis techniques, and related quantitative aspects of testing. From 
the teacher advisors* point of view, none of these technical developments had any direct 
relevance for classroom assessment, ktrgely because they required a level of training 
and time-commitment beyond that of virtually any practitionero Several members of 
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the technical p^n^ls^ urged inclusion of some of this technical materiial in a simplified 
form. We decided not to discuss complex technical advances in any depth, but to include 
a list of resources at the e^id at each chapter that the reader could consult for extended ' 
treatments of these topics. In retrospect, we may have erred oh the side of excluding 

information on basic quantitative aspects of. testing, such as item analysis procedures, 

*^ • ' 

but, given the intended audience for the Handbook and the context in which it is to 
be used, the decision to keep discussions of technical measurement topics to a minimum 
seemed warranted. 

finally, many more specific issues were encountered in° the process of writings 
indi^ridual chapters. FoV example, when writing Chapter 6, on formulating /learning goals 
for students, members of the Teacher's Advisory Panel indicated strong dissatisfaction 
with traditional prescriptions for formulating objectives, which require teachers to 
Specify not only the knowledge and skills students are , to acquire or apply, but the 
conditions under . wnich student learning will be assessed andtthe criteria against which 
student performance on assessment tasks is to be evaluated. Drs. Millman jand Hambleton 
also suggested *hat including performance criteria in goal statements Was impractical, 
and not necessarily desirable. Several drafts of Chapter 6 w,ere required in' order to 
establish guidelines for writing goals that appeared both sound and feasible. 

The issue of how best to write learning goals was merely one of many that were 

dealt with on a chapter-by-chapter basis, and that often led us to revise substantially 

♦ 

what we had initially written. In fact, each of the chapters in the Handbook was 
revised at least once, and in many cases, two or three times, to assure that Its contents 
were treated appropriately. C itiques provided by Dri. Millman and Hambleton were 
particularly helpful in this process. 
Distinctive Characteristics of the Final Product 

The distinctive characteristics of the Handbook have been suggested in the 
previous section. These characteristics are summarized and made explicit below. 
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(1) The" Handbook sets forth in reasonable detail a goal-base<i approach to 
integrating teaching and testing,; A conceptual perspective on the nature 
ot learning is introduced in Chapter 2 artd used as a basis for^^rganiz'ng 
Chaptef 6, on formulating learning goals, Chapter 7, on matching instruction 

' to goals, Chapter 8, on matching' assessment to goals, and on major sections 
of ^Chapters. 10, 11, 12, and 13. We believe the idea of using learning goals as 
a basis, for linking Instruction, ^issessment, and the use of assessment 
Information is both theoreticaUy sound and responsive to the general trend 
in education to require both teachers and school districts to make public 
the learning outcomes, students are expected to achieve in courses and 
programs and to report evidence^on outcome attainment to» students, parents, 
and members of the community. 

(2) Having been developed under the gu^danc^ of 12 high school teachers, the 
Handbook reflects sensitivity to contextual factors that Influence teacher 
pj^actice- Chapter 3, for example, includes a descriptidn of the influence 
that student characterisitics have on teaching and testing, and the implications 

, that different instructional i^odels have for these processes. More generally, 

/ 

the guidelines an^djnumerous Illustrations contained in the documeht have 

been carefully developed to r,eflect high, but attainable standards for teachers' 

7 - ^ 

professional perforrtiance. 




(3) The Handbook has been pilot tested in the context of^aTschool-based inservlc* 
program, and carefully reviewed by riationally-known technical conpultapas. 
Both Its content "-..id form have been Judged to be of good quality and 
appropriate for the Intended audience. 
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DEVELOPMENT AND PILOT TESTING OF. A STAFF DEVELOPMENT 
PROGRAM BASED ON TB^ HANDBOOK 

During the falj of 1983. a smul scale, school-based inserviee program was 
developed in ' collaboration with four members ot the Teachers' Advisory Panel. The 
Panel as a whole had completed its work in guiding the design of the Handbook, but 
these four members expressed a special interest.in the project and volunteered to assist 
in developing and pilot testing an inserviee program around' the H'andbt>pkx We refer 
to'^ these teachers as "lea \ teachers." ^ 

^ The inserviee program was intended to provide an opportunity for lead teachers 
to work with colleagues for two fuU days in their home districts on the integration of 
teaching and testing. More specifically, the program was desi^d to permit^rtieipants 
review ahd discuss, the ideas and methods contained in the Handbook and to use this 
content as an aid in developing an instructional jinit. appropriate for their setting 'and 
rieedsf that made clear connections between instruction and assessment. More detailed 
, information about the inserviee program is offered below. ^ 
Purposes to be Served by the Pilot Test ^ 

The pilot test of the Handbook and the related inserviee program vyas designed 
to serve the following purposes: 

1 

I • 

• , . To assess. high school teachers' reactions to the Handbook as a resource for 
integrating teaching and testing; 

To assess high school teachers' reactions to the inserviee program developed 

around the Handbook. ' . i ■ 

I 

I • 

. ,To assess the reactions of lead teachers to the inserviee program and the 
* role they carried out in it; and« 

« ■ * 

To serve as a basis for the design of instruments for, obser^ving teaching- 
testing practices, and for assessing their .effects on student attitudes. 



..Sample ' , ^ ^ 

Schools. Pour comprehensive public high sclipols participated in the study. " One 
school wai in southwestern Washington. The^ other three scdwols were in weste'rn 
Oregon. .The schools ranged in siz^ from 460 students to 1.800 students. Three schools 
served students at grades" 10 to 12; one spanned^ grades 9-12. ' 

tead Teachers: A lead teacher from eacfj school served as faciUtator of the 

inservice program. The lead teachers met for -two days' during the fall and winter of 

1983 to prepare for their roles as facilitators. T-his preparation imrolved shaping agendas 

for the work sessions, reviewing relevant content, and drafting illustrative teacjilfig and 

testing material for a particular unit of 'instruction. In addition, one of theaitea5shers 

met individuaUy with project'staff in early January to discuss content and Wcedures 

about which he had questions and to refine the iUustrative teaching-testing ma>erials 

he had developed. The , lead teachers represented .the subject areas of . science, social 

% - • 

studies, and foreign languages. . 

Regular Teachers. Two teacfiers from each of the four soi|K>ols participated with 
" the leid teacher In the inservice program. In each actjool. these teacWrs were selected 

« * 

by the lead teacher and the .principal because of their interest in the proposed content 
of the prograni. Participation ^as voluntary, these teachers represented the following 
subject areas: sdlence, art, social studies. French, mathematics, and English. 
• Unfortunately, due to illness one teacher from each of twq schools was un.able to attend 
the second day oX the two-day program (the program is described later in this report). 
The total number of regular teachers who participated fuUy in the program was thus 6. 

4 

^poject Staff. A member of the project staff participated In each of the two 

f 

all-day .inservice sessions in the four schools. His role was to provide a context for 
the Handbook and the inservice program; to clarify Issues of. content that arose from 
teachers' reading of th| Handbook and their effort to translate ideas from t4t<^ 4^andbook 
into teaching-testing material; and to support the lead teacher in any -vay that seemed 

17 

13 . r 



appropriate a^ the lead teacher Buided his eoUeagues through the designated activities. 

Prindpato. In three of the four schools, firojeet staff thef personally with the 
principal in the Ute faU or early winter to discuss the inser.ice program. AU principals 
had been informed a66ut-the program in the spring of 1^83. but a face-to-face meeting 
was considered desirable to firm plans and to assure the principals' support. In one 
school, the principal gave clear verbal support, by phone, and did nbf feel a need for 
a personal meeting; All principals received the draft Handbook and a -synopsis of the, 
Inservice program. . ' 

PrlncipaU had no' role to play in the field test other than to communicate their 
support of the program to teachers who were participating In it. One principal, however, 
took the initiative to attend about 15 minutes of the first day of the inservice session. 
He suggested that he took this step to increase his understanding of the program and 
to actively demonstrate his inierest in the work being done. ' • 

Expected Outcomes ' . ^ ^ 

# m 

The original design for the Inservice .program set forth a wide range of 
expectations. Teachers were expected to read and discuss a large portion. of the 
Handbook, and to develop an instructional unit that included not only statements of 
expected learning outcomes, . tests, and descriptions of anticipated 'uses of test 
inforrpation,' but- information on teaching methods, ancj informal, day-to-day assessment 
tactics. Dj^ing the first work" session at the first school to field test the program, it 
became apparent that the focus of the work sessions needed to be narrowed somewhat. 
Both of the regular teachers needed more time, then was allocated to formulate expected 
learning outcomes according to the framework proposed in the Handbook. (An EtigUsh 
teacher in th^fe session had particular- difficulty with this task because, by her own. 
account, she was wfaccustomed to specifying student learning goals or objectives.) The 
teachers in this first session also indicated that they were most interested in thinking^ 
abou^ ways ol strengthening their approach to testing outcome attainment. The original 
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plan called for a description of instructional strategies And informal assessment 
procedureis. but the teachers did not assign these tasks as high a priority as test 
development. They noted that they had received training In instructional strategies, 
but had little formal bacltground in testing. Therefore, given the time constraints, they 
suggested that they focus on instriictional strategies only to the extent that they had 
concerns about them. Thi?^88med^ reasoi^ble to the 'lead teacher and the project staff 

member, eacM^of whom pointed out that issues of instruction could be addressed later 

■> 

in the context of a discussion of the instructional implications of test results. 

Based on the first two or three work sessions, in which v^ous modifications in 

) - ^ ^ -I 

scheduling and content coverage were tri^d and evaluated, a more nafrowly defined set 

of expectations ^as establisf^d that seemed ,more feasible to achieve in the time 

available/ The revised set of tasks that participants in theCprogram were expected to 

complete is presented in Display 1. v . 

Training Procedures ' . - . 

The inservice program ^as intended to' provide an opportunity for teachers to 

•» ' ■ * ' • . • 

work both together and individually on the integration of teaching and testing. More 

specifically, the program was designed to permit participants to review and discuss the 

ideas and methods contained, in the Handbook and to use this content as an aid in 

* •• , ; — — 

. carrying out the tasks identified in Display 1. ' ^» 
Two full-day work sessions were allocated for the program in each building (a 
total of eight sessions were thus held). These took place between .id:January and 
early March, 1984. These sessions began at 8:'30 and* adjourned at 3:30. For each of 
the major tasks to be accomplished (Display 1), time, was. given to review relevant 
chapters of the Handbook and to discuss issues* raised through the reading.' Each 
participant received the Handbook at least 3 weeks prior to the sessions.^ so some 
familiarity with its cont(ent and organization was assumed. Following a discussion of 
relevant content, participants had an opportunity to work on a designated task, 
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. DISPLAY I 



PILOT TEST OF f HE HAMDBOOK AND 
RELATED STAFF DEVELOPMENT PROGRAM 

January, M/ireh» 1984 



Tasks to hi Completied Through the Two AU-Day Work Sessions, 
Supported by About Two Hou^s^f Time Outside of the Work Sessions 

• ' * 

WriiLe learning goals and objectiives for an instructional unit that you plan on 
teaching between March 1 and May 1 of this year. Classify these '^oals and 
objectives according tp the framework provided, and indicate the relative emphasis ^ 
each is to receive foi^purposes of instruction and assessment. 

Develop a unit test or comparable assessment procedure that matches unit goals 
and objectives and the emphasis that is to b& placed on each. Use the guidelines 
provided in chapters 9) and 10 of the Handbooli to help asure that yout, test meets 
standards of technicc^ adequacy. « 

Set a standard of acceptable test performance for the class as a whole. (You^ mBf^ 
also set separate standards for individual students. If so. provide a rationale for 
this.). Indicate the type of options you will pursue if these standards arc not met. 
Specify how many class days, if any, you are willing to devote to helping students 
^cor.'edt learning deficiencies revealed through the unit test. Also specify any 
provisions that will be made for retesting students after colrrective instruction has 
been completed. Finally, de5crit>e what type of enriching instruction, if any, will 
be given to high-achieving students, while others are receiving corre^Stive instruction. 

Describe procedures for reportiog and reviewing test results with the class as av 
whole, and .for giving feedback to individual students on their tes^ performance. 

specify how test results will be used in calculating a grade for the, marking period, 
quarter, or semester. 
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Participants worked on thik tasks individually, but then shared work products with 
their colleagues. After a review and discussion of the products, some time was allotted 
for individuals to refine jtheir .work,*.*lthough participants were expected to fine-tune 
^ n)aterial outside the context of the work sessions. / 

• The. lead teacher in each building and Vak project staff member (the 'same staff 
person attended all work Sessions) guided the sessions. The kin^ of partnership that 
. emerged between lead jteacher an() project staff varied across sites. In twS of the 
sites, the lead teachers were accpmplished leaders of inse^v^ce ^ograms in their district 
and had also invested considerable time in preparing illustrative units inteiided to 
highlight the link J:>etween instruction and assessment.. In these two schools, both the 
-project staff rnember and the 'lead teacher worked togethe^" to explain Expectations, 
clarify content, and respond to concerns about the applicability of ideas and procedures 

, . to particular topics" and classes. 

X . • • - • 

^ Two of the lead teachers assumed a less active posture in the work sessions. 
They deferred more to the project staff member both with respect to explaining content • 
,/\ and commenting on teacherS' work. ♦ ^ 

Reactions of Participants to the Handbook and to the Inservice Program 

The main tool used to ^ assess participants' reactions to the Handboo k and the 
inservice program was an 11-item questionnaire. This questionnaire was given to the 
regular teachers at the end of the second day of t!he wqrk sessions. Two jf the eight 
regular teachers were absent for the se.cond'day and did not receive the questionnaire. 
Informal phone conversations rev saled that these, teachers had poiitiv?^ attitudes toward 
bo^'i the Handbook and the t'lr^t day. of the program, but no formal assessment of their 
reactions was conducted. 

A report of responses to each of the eleven items on the questionnaire on the part, 
of the si)(f teachers who completed it is presented below, fc 'lowed, when appropriate, by 
a brief commentary. As will' be seen, the general response to both the Ha ndbo ok and 
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'erviee program was "quite positive. 

QuestioQ 1 ; About how much time, if any, did you spend reading, the Handbook 
before the work sessions? >i 

. Mean response: 1 hour, 18 minutes ' 
Range: 30 minutes to 2 hours, 4S minutes ^ 

" Teachers had been asked to read the first several chapters in the Handbook and 

chapter 16, on formulating learning go&ls. The reading was intended to convey the, 

overall approach to integrating teaching alid testing with which the participants would 

be dealing, and to prepare them specifically to write learning goals for "triil" 

instructional units. The personal impression of the staff mek^^r attending the sessioi^s 

was that these sel^-reports^^overstated the'edtual time invested. When asi ed to comment 

on the framework for thinking about learning goals described in the Handbooks for 

example, few of the participants could deadly recall its b^sieyfeatuT'es. ^orhaps this 

was because insufficient structure was provided to guide the teachers^ reading. 



( 



Question 2 : About how much ti; <e, if any, did you spend reading the Handbook 
between the first and second day of the work sessions? 
■ > ^ ■ " . 

Mean response. 1 hour, -35 nirutes 

Range: 45 jnin^ites to 4.5 hours - ' . 

Teachers were expected to read chapters on the use of assessment information 

(Chapters U-13} between the first and second sessions. Most seemed to havie done 

/ 

this and were able to point out at letist one or two ideas they f'^und useful, or. at least 
noteworthy. ' 

Question 3 ; Please describe in a brief paragraph how you used the Handbook 
during the work sessions, • \^ ^ 



Representative teacher responses t^ question 3 are presented below. (They have 

\ 

been paraphrased slightly, for ease of reading). ^ 

♦ • 

" Teacher 1: I used' it primary to give examples of things that I might try. 

Teacher S: During the first day, the handbook clarified vocabulary and content 
questions. During the second day it provided actual help for tes^ writing and 
'evaluation. / 

V Teacher 6: FOr clarification - so I cpuld be sure I fulfilled my responsibilities 
- also for intereJt - I was intrigued by the concepts. 
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Question 4 ; Which part or parts of the Handbook did you find qiost helpful?/ 

Teacher 1: The parts of the handbook involving designing tests - particularly 
the "memory" and "use" type of questions. The methods for. evaluating by 
observation were very useful. . ' ' » 

Teacher '2: The- areas on writing objectives and v^riting test question^. f 

Teacher 3: Chapter 7 was a good summary of techniques used to teach. Chapter 
9 - the most useful - specific examples of types of test items. For me, all the 
theorizing in 'the book could have been reduced' and extra emphasis placed here. 
Qhapter 10 also contained good information on testing. 

• ! , / « 

Teacher 4: Chapter 9 was probably the most helpful. 1 also looked closely at 
chapter 12, aboul what to do with students who -fail exams. . 

^ Teacher S: (parts dealing with test item writing, reviewing >tests, and' those 
students who donU grasp'^^naterials. I found Ihe examples « in each section to be 
extremely helpful!! , ; 

• / 

Teacher 6:^ I- found several parts helpful - level of performance and its tie with * 
Bloom's taxonomy, securing motivation in a low interest student, assessing why 
J a student is failing are just three examples. They y^eie helpful in that they 

'> ♦ offered an intelligent, practical approach to the problem. It. is ver. difficult to 

answer this question as I found so much .that was helpful. 

* . • ' . ■ r . 

/ - • 

Question 5 ; Which part or parts of t^e Handbook did you find leas^ helpful? 
•How do you think chese parts could be improved? ' 
. • • - i 

• Interestingly, three of the six teachers left this blank. One who did respond said 

at first she thought, there would be too much to read, but then found the assignments 

acceptable. Sh6 concluded, "I have- no suggestions to make." 

Another respondent said the comparison between the framework for viewing 

learning goals presented in the Handbook and Bloom's taxonomy was least helpfuk.be^ause 

he "...didn't grasp as^much of this as I'd like." Finally, an art teacher stated that the 

section on. writing objective test items was not very relevant to his class, thouph he 

indicated that the material on product-related assessment was pertinent. \ 

^ Question 6 ; In your view, how much value has the Handbook been as an overall 

' resource for planning instruction, developing interests, and using 
test information? > 

Mean response: J.l on a 4 point scale in which I = no value and 4 = a great 

deal of value 

Range: 2 to 4 , ^ ■ 
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Question f t In what\ ways, if any, do you feel you benefited from- the work 
sessions on integrating teaching and testing? ^' 

Teafiher 1: Th^ assistance of the instruetor(s) was very beneficial in helping me 
design my tests toward what 1 teaph, and my teaching more directly tied to what 
I. want students to learn. 

jTeaeh^ 2:. The time to wiUe goals and tests is very limited. This time w|is ' 
^valuable for that. I really felt that having a resource person here was very helpful. 
* • ' . If 

Teacher 3: I feel that I benefited from some of the ideas of ways to te^ for 
different learnings > giving different emphasis to different styles. Since I use 
, a text-related standard test, I think I am. better able to evaluate good and bad 
items in the tests. 

Teacher 4: I am more aware of the expectations that I have of students - also 
the .limitations. ' . * " . 

Teadier 5: It gave me the opportunity for 2 full days of, work. Glen*s guidance 
and the time were the most valuable.' 

Teacher 6: I gained a new perspective which enabled me to improve 'my testing. 
With an adequate background in the subject area, and a' few years teaching 
' experience, it is easy to get In a "rut" that repeats weaknesses. This session 
influences that type of weakness. 

Question 8 ; Was there anything about the work sessions that was disappointing 
or negative? How^ if at all, could these'aspects of the work sessions 
be impr^red? 

Teacher 1: No - » 

Teacher 2: \ wish we could have been away from the school area. Too many 
Jlstractions.' 

Teacher 3: I would have like a more structured approach. I think there was 
too much theorizing. I also find it very difficult to try to formulate ideas in a 
setting where I almost felt like I was being "watcjjed;" I prefer to work alone 
- I would have preferred to concentrate on my own goals - alone. 

^ Teacher 4: It was a one-on-one situation, so the immediate response was great. 
However, time was of the essence^ - need^ more time - another day. ' 

• Teadier 5: The first day's setting was not a working setting for me. T^ noisy, 
and not enough time to work alone. 

Teacher 6: I can offer no suggestions. 

The Issues of setting and time were^tipparent to the project staff member who 
attended the sessions. Four Individuals working together in a relatively small room for 
two six-hour »?ays represented an Intense work environment. And, even thou^ the 

' ■ - ■ ..^ 
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number of tasks had been reduced after the firsts Inservice session, it was difficult to 

achieve all the outcomes desired in the time allotted. ^ 

Question By the end of the second day of the work sessions, how far alon^ 
/ weTe you toward completing the instructional unit call for in the 
project? ' 

Mean response: 3.2 on a four-point scale in which 1 » *1 didn't start the unit" 

and 4 = completed nearly all of the unit 
Range: 2 to 4 - 

• . , «* 

Question 10 ; How, If at all, was the approach to unit development introduced 
in the work sessions deferent f^om the approach you normally 
* " follow? Was there anything about this approach that you especially 

liked or'disUked? 

Teacher 1: Planning the unit around specific outcomes and planning the'' test 
« based more closely on what students were supposed to learn gave a truer picture 
^ of what actually was and was not learned. 

Teacher 2: The approach was low key, and 1 wasn't sure about the direction 
we were going at first, hut as we worked along 1 felt' more organized. ^ 

Teacher 3: I don't want to imply that I don't "organize my thoughts before 
teaching, but teaching elementary levels of French involves so many minute 
components that the idea of having to write goals 'and objectives for each one 
is overwhelming. Therefore, some of the writing of goals is a waste of time to> me. 

Teacher 4: I normally teach in units. It was ok. 

Teacher 5: The approach was more detailed, with a definite beginning and ending. 
Things are usually done day to day for me. I enjoy^ this approach and will 
attempt to u^e it regularly. " 

Teacher 6: It was a more "planned" approach. I liked feeling that I had produced 
^ a good test - it gave me more enthusiasm and energy as I taught the unit.. 

s 

Question 11 ; How much help were the work s'essions to you in strengthening the 
links between your teaching and testing practices? 

Mean response: 3.1 on a 4-point scale, in which 1 « no help and 4 = a great deal 

of help. 

Range: 2-4 

On th« whole, then, the reaciions to both the Handbook and the inservice program 
certainly were encouraging, 'judging from responses to the questionnaire it appeared 
that the work sessions assisted teachers in thinking about instruction and assessment 

i 

more systematically. However, no objective evidence on the effects of the program 
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on teacher practice or on students was gathered during the pilot test. As is indicated — 
in the last part the report, however/ evidence of this kind on the effects of the 
inservice program is currently being collects as part of a related project sQpported 
by the NIE, . 

Reactions of Lead Jeaehers . ^ ^ 

i - ' 

" Reactions of lead techers -to the inservice project were assessed informally after 

each of the school based sessions, and then again in June' at a luncheon meeting. 

With respect to tl^e content of the program, these teachers Indicated that th^ 

refined a number of their id^aS and practices regarding instruction and assessment based 

on their in ^Ivement in the project. Here are some representative conments on this 

topic. (They are paraphrased slightly for ease of reading). 

Teacher A: The project has made me^ think a lot more about what I want to 
focuf on and then test. I never thought much about giving different weights to 
test items, for example, based on the emphasis I've given particular ob^ec^ves 
in class. Also, the idea of paying more attention to what test results imply 
about next steps to take in instrucUon stays with me. ..I've been trying some new 
ways of helping kids to figure out why they got some questions wrong on a test.^ 

Teacher B: Letting students in on what the learning goals are seems important. 
I do this in a clearer, more direct ^ay now. I also look at the goals when 
making tests, whereas before I had them more in the back of my mind... I think 
thinking about goals more Aends to push you into higher levels of learning... I 
also am giving more'^uizzes and doing more checks during units on how students 
are grasping the material. ^ 

With respect to the processe^ used in the program, the lead teachers generally 
felt satisfied with, the work sessions and their role in them, but expressed these 
suggestions: (These, too, are paraphrased slightly). 

Teacher A: "Have the inservice earlier in the year, maybe October or early 
November. This will help people think more about the course as a whole and 
also let them try out new ideas over a longer time period. 

Teacher B: If possible add a day or two to the inservice so that topics could 
be dealt with more fully. 

Teacher C; Consider having teachers from the same department meet together. 
While there is some value in having people from various fields get together, 
teachers from the same department would be able to deal more effectively with 
substantive Issues. 
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Teacher D: Build in more "accountability" between lead teachers and their 
colleagues. Lead teachers need to follow-up on what their colleagues do after 
the work sessions and give support and >assistance as needed. This idea was 
discussed in the project, but a system wisn*t set up to help it^et done. 

This last point seemed particularly important, though no easy resolution of the 

issue seemed possible without greater involvement of school administrators and perhaps 

more released time, or more flexible teaching schedules, for teachers, participating in 

the project. As with many inservice efforts aimed primarily at the classroom teacher. 

the implications of the program for school policy and management issues were large. 

Many of these issues are addressed in the implementation manuals, which are described 

in the next part of the report. 

Use of the P.Uot Test as a Basis for Designing Research-Related Measures 

During the pilot test a member of the staff of the project began work on a 
related project sponsored by the Center for Educational Policy and Management (CEPM) 
at the University of Oregon, with funds provided by the N IE. This project Was intended 
to serve as a sequel to the handbook>development project. Its aim was to refine the 
staff development program being pilot tested and to conduct research on the effects 
of the program on teacher practic^^^h^ student attitudes. The CEPM project also was 
intended to yield information /on administrative policies and procedures that affect 
classroom practices. The CEPM project is described more fully in the last section of 
this report. 

The pilot test ^of the staf i <}evelopment program described in the proceeding 
pages provided a context within^A^hich to devlop the instruments called for in the CEPM 
project. These instruii(ents included (a) procedures for observing a teacher*s approach 
to giving feedback to the class on its test performance; (b) a guide for interviewing 
teachers before and after the observation to obtain supplementary information on test- 
related practices; and (c) a questionnaire for students on their attitudes toward 
coursework, testing, and grading, and toward themselves as learners. 
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" Each of the^ four lead teachers Involved In the pilot test kindly agreed to allow 
project staff to observe their classes on. days when tests^^were being returned and to 
videotape some of these lessons. This helped immeasurably in the development of the 
classroom observation instrument referred to above. Two of the lead teachers, alon^ 

« 

with other participants in the inservice program and several teachers from schools not 
previously involved in the project, agreed to pilot test the teacher and student 
questionnaires prepared for the CEPM project. Thfe pilot test of ^he inservice program 
thus provid-jd a basis for developing the measures needed to carry out research on the 
effect^ of this kind of program on teachers and students. 
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PREPARATION OF STAFF DEVELOPMENT MANUAI^ FOR 
TEACHERS, LEAD TEACHERS AND PRINCIPALS, AND SUPERINTENDENTS 

Experience gained through the pilot te^t of the Handbook and the 2-^ay inservice 

program on its use provided the basis for the preparation of these manuals. They are 

responsive to the e;cpressed needs of teachers who engaged in the inservic,e program; 

they are responsive to the experience of the lead teachers who conducted the program, 

and the suggestion of principal? who had to approve and support the program in their 

buildings; and they anticipate the essential role that Superintendents play in the decision 

« 

of a school or district to initiate a staff development program for its teachers that is 
intended to change the nature of school practice and i;equires the expenditure of 
considerable time and energy to do so. v * 

In combination these manuals are intended to provide a "stand alone" training 
system that districts will be able to purchase at low cost and implement with' their 
own personnel. The training system rests on the central assumption that with adequate 
resources a principal, and a staff member selected by the principal to serve as a "lead 
teacher" for a department, can function effectively as a training team for faculty within 
the teacher's depar/ment. The instructional resources needed to conduct the training 
program are carried in both the Resource Guide for Teachers and the Planning Guide 
for Lead Teachers and Principals. The information needed to explain the program t,o 
a Superintendent, and to convince him or her of the value t9 be gained through it in 
relation to the cos^s involved, is carried in the Program Guide For Superintendents . 

This section of the final report contains brief descriptions of the content and 
organization of those manuals, and how they have been developed. It also contains a 
description of the field trials alnd refinement process that Is planned for the manuals. 
AS the manuals presently stand, only parts cf the first half of the Resource Guide for 
Teachers (the part 4^ the Guide that contains a two-day BASIC training program) has 
been used and reviewed by teachers. ^ 
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The Resource Guide for Teachers 

The staff development program is divided into a two-day BASIC training program 
and a three-day ADVANCED training program! The BASIC program, fojju^es on the 

* • 

• integration of teaching and testing practices, in the context of instnietionGl units, and 

* * ■ 

is linked tightly to the content of the HaYidbook developed through the project. It "STso 
parallels closely the two-day- inservice prog*am that is described in the previous section 
of thi$ report, and the two-^day training program that constitutes the treatment condition 
in an ongoing research study funded through the Center for Educational Policy and 
Management at the University of Oregon (see the last section bf this report). 

Th^Ai;)yANCEO program focuses .on the integration of teaching and testing 
practices in the context of a course of study, and addresses the issu^ of how faculty 
^ within a department articulate learning goals across courses jwithin a program of study. 
The advanced training prcgrartP also ties closely tp the contehf. of the Handbook, but it 
is net limited to this content. Appendices included in the Resource Guide , and information 
brought to the program by the principal, supplement the Handbook in topic arfas that 
go beyond it. * . 

In combination the Handbook, , the Res^ource Guide and the infbrmation that lead 
teachers and principals are able to bring to tfte training sessions are intended to provide 
all of the instructional resources needed to prepare *teachers to effectively integrate 
teaching and testing practices in the courses they teach. An outline of the content of 
the Resource^.Guide is provided in DISPLAY 2. For each day outlfned in this display, 

~ w 

the Guide contains a suggested^ work plan fc ' -5 day and an instructional unit fj>r each 
of the topics listed. ^ i \ • . 

The Planning Guide ^or Lead Teachers and Principals 
« ,7he Planning Guide parallels the Resource Guide in its organization and content. 

There is an introductory section that"^utlines the roles and responsibilities of lead 
teachers and principals in the training program, and the philosophy underlying the 
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DISPLAY 2 

TABLE OF COHTEKTS FROI^ tHE RESOURC^rtSDIDE FOR TEACHERS 



A Note to* Teachers ., 

IHTRODUCnOK , 

PART L .THE BASIC TRAINING PROGRAM 
Overview^ 



Quteomes to be achieved 



1: Establishing learning goal;; for a UNIT of instruction, 
designing an assessment plan, and developing measures 
to assess student mastery of the learning outcomes desire^ 



Day 2: Setting "standards for goal mastery, deciding how performance 
standards are fo be applied to' slow and fast learning students, 
and giving feedbpick to students on test performance that 
highlights what has and has not been mastered 



PART n. THE ADVANCED TRAINING PROGRAM 



Overview v< 



Outcomes to be achieved 



Day 3: Reviewing the effects on student learning df work 

accomplished in the BASIC training program, refining 
the framework used in the basic program for thinking 
about learning goals, and articulating UNIT goals with 
COURSE and PROGRAM goals .\ 



Day 4[yDesigning an assessment plan for a course of instruction, 
linking grading. standards for a course with performance 
standards for goal attainment, and reaching departmental 
and school-wide agfeement on grading standards, including 
standards for slow and fast learning students 



Day 5: Using information about goal attainment to plan a 
COURSE of instruction, guidelines for managing 
goal-based instruction wi^ students who are working 
at expected levels of accomplishment, ^nd guidelines 
for students Who ar&jworking toward learnin{f goals 
that deviate appreciably from expected lev^els of 
accomplishment ^ 



APPENDICES 



A. An Introduction to Mastery Learning 

B. Notes on the Articulation of PBOGRAM, COURSE, and UNIT goals 

C. Illustrative Learning Goals for a Course of Instruction 
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training program generally. There also W a discussion of the key role that a 
superintendent needs to play in reviewing and appjipving the program, and in either 
lending support or giving leadership to its implementlSition. . ^ ^ . 

The core of the Plarinihg Guide , however, is the NOTES AND RESOURCES for 



lead teachers and principals that accompany each day of the training program. These 
■ parallel and build on' the instructional • resources presented in the teacher's Resource 
Guide , and are intended to provide the whereWlthall the lead teachers and principals 
need to function effectively as facilitators of the training program. In this regard, 
the recommendation Is made in the Planning Guide that the lead teacher/principal teams 

^ 

who are to facilitate the training program within a district engage in at least a week 
of intensive preparation/study before the training program Is undertaken. It is 
recommended that this be done under the tutelage pf someone in the district or a 
nearby college who is knowledgeable about tests an^urrent testing In the schools. 
The Program Guide for Superintendents 

The Program Guide serves a very different function thaji the t^o manuals that 
have been descrlbtd previously, though it Is viewed as being complementary to the other 
manuals and essential to the program as a whole. The purpose of the Program Guide is/ 



to inform the superintendent, other administrators and members of the 'local Board of 



Education about the benefits to be gained by integrating teaching and testing practices 
within a district, and to orient them to the training program as a means of doing so. 

S I 

An overview also is provided as to what the integration of teaching and testing practices 
means operMionally, the implications of such practices for staff development and the 
cost oj implementing the training program. The preparation of the Program Guide is 
founded on the assumption that a change in th.e nature of schooling of the kind that 
comes with the effective Integration of teaching and testing is one that must be 
considered carefully by everyone concerned before a decision to make such a change 
is made. It also rests on the fact that in smaller districts resource expenditures 
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of the kind required to implement the training program are always subject to the review 

and approval of the superintendent.' * 

Plans for Field Testing and Refining the Staff Development Manuals 

As indicated previously none^ of the manuals 'have been field tested in their 
present form. Foui'^of the six instrueji^pnal units contained in the 2-day BASIC training 
program were used in the pilot test of the Handbook, and currently are being used in 
the ^EPM sponsored -experiment described in the^ next section of this report, but that 
is the extent to which ihey have been subjected to the realities of field use. An 
extensive program of field review and testing is planned for the months ahead, hovyever, 
and revisions/refineB^erits will be made in the manuals oh the basis of them. The refined 
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versions of the manuals will be submitted to the KIE in conjunction with the final report 
on the CEPM spohsored experiment in this area that currently is underway. The field 
review arid testing program that is planned is outlined in DISPLAY 3. 



/ DISPLAY 3 

PLANS FOR FIELD TESTING AND REFINING THE STAFF . ' 
I DEVELOPMENT MANUALS . 

The Resource Guide for Teachers 

. A critique by teachers and lead teachers participating in -the CEPM experiment (Winter and 

Spring, 1985). 

♦ 

2. ' A* critique by members of the Valley Education , Consortium Planning Council, which includes five 

secondary principals, four elementary principals, and four district curriculum coordinators (Spring, 

- 1985). ^ . 

i. > . ' 

3. A critique 'by members cff the Valley Education Consortium work grouj^ on Program Evaluation, 
Troubleshocting and Improvement, which includes four' superintendents, four curriculum 
coordinators and for purposes of the review four ieachers who have been nominated to serve as 
lead teachers in the superintendents* districtls (Spring, 1985). ^ . ' 

4. A critique by lead teaeller and principal teacns within VEC districts who have^ completed a week 
' of training with the Resource Guide and Planning Guide In 4>reparatlon for implementing the 

training program in thel^ schools during the 1985-86 school year (late Summer. 1985). 

5. A critique by teachers, lead teachers^ and principals who participated in the .training program 
In VEC dislriets (late- Fall, 1985). ' 

^^^..^-.....^he Planning Guide For Lead Teachers And Principals ^ 

1.' A -jritique by lead teachers and* principals participating in the CE.PM experiment (Winter and 
' * Spring,' 1985). 

2 A critique by members of the Valley Education Consortium .Planning Council (Spring, 1985). ^ 

3. A critique by members of the Valley Education Consortium work group on Program Evaluation, 
Troubleshooting and Improvement (Spring, 1985). 

4. )k crittque by lead teacherrand principal teams within VEC districts who have completed a week 
'jot training with the Resource Guide and Planning Guide In preparation for implementing the 

training program in' their schools during the 1985-86 school year (late Summer, 1985). 

5. A critique by teachers, lead teachers and principals who participated in the training program 
in VEC districts (late Fall, 1985). 

The Program Guide tjse Superintendents 

1. A critique by members of the Valley Education Consortium work group on Program Evaluation, 
Troubleshooting and Improvement (Spring, 1985). ^ 

2 A critique by lead teacher and principal teams within VEC districts who Rave completed a week 
N* of training with the Resource Guide and Pla nning Gui<jfe in preparation for implementing the 
training program in their schools during the 1985-86 school year (late Summer, 1985), 

3. A critique by superintendents of the VEC districts in which the traming program has been 
Implemented (late Fall, 1985). 
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RELATION TO. CURRENT WORE SUPPORTED BY THE NIE , 

As indicated earlier in the report, the NIE has provided funding for a research 
project, sponsored by the Center for Educational Policy and Managemejit (CEPM) at 
the University of Oregon, that builds upon the handbook development project. The 
CEPM Project extends the original work in two main prospects. First, it is intended 
to produce more information about school and departnient policies that influence how 
teachers carry out instruction and assessment in their classrooms. ^We knew from pur 
experience in the Handbook-development project and our work with schools generally 
vthat the curriculum adopted bv a school, the textbooks in use, the approach to teacher 
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'supervision that is taken, the availability of test-item banks and ^test-scoring machines, 
. and other factors that lie outside the control of the individual teachers influence what 
teachers do in the classroom. The CEPM Pfoject has provided an opportunity to explore 
connections, between aspects of the school context jand classroom practices. . 

VThc CEPM Project also has provided an opportunity to refine the inservlce program 
that was pilot tested in early 1984 and to conduct research on the effects of the 
refined program on teacher practices and student attitudes. The research follows, an 
experimental design in which science teache^ from five high schools participate in an 
inservice program, and science teachers from five other high schools serve as a control 
group. Data -on teacher practices and ^tudent attitudes are collected on a pre and « 
post-treatment basis. The progress that is being made on the CEPM study is reported 
periodicaUy to the NIE. , 
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INTEGRATING TESTING' AND TEACHING: 
AN UPDATED REVIEW OF THE UTERATURB 

■ , ■ \ 

In recent years, educational researchers 6nd practitioners have paid increasing 
attention to the relationship between testing and teaching. The growing interest in 
this topic is signaled by a number of conferences and publications, mcu notably the 
review of the literature on integrating assessment with instruction prepared by Herbert 
Rudman and associates (1980) at the Institute for Research on Teaching; the conferences 
sponsored by the National Institute of Education in 1978 on research on testing, teaching 
and learning, and on achievement testing and basic skills; the monograph series. New 
Directions for Testing and Measurement, published by Jossey Bass; the forthcoming 
spring issue of the Journal of Educational Measurement , which is devoted to the theme 
of linking testing and instruction; a series of reports on the use of standardized tests 
in American schools, produced by a team of researchers at Carnegie Mellon University 
and the University of Pittsburgh (Resnick, 1981); and a comprehensive survey of test 
usage in schools and district^ completed by investigators at the Center for the Study 
of Evaluation at the University of California at Los Angeles (Burry et al., 1982). 

The present review has been prepared as backdrop to the development of a 
handbook intended to help teachers better integrate assessment and instruction. Special 
attention is paid in the review to information on the testing-teaching linkage at the 
high school level, since the handbook is being designed for use by teachers at grades 9-12. 

The review is based on findings and conclusions reported in the conference 
proceedings and publications referred to earlier, and on recent studies not cited in 
these documents, in relation to the following questions: 

(1) What types of tests and assessment procedures do teachers typically use, 
and ^or v;hat purposes? 

(2) What are teachers' attitudes toward and concerns about testing? 



(3) What do teachers generally know about testing, and what aspects of testing, 
or test usage would they like to know more about? 

(4) What advances in the technology of testing, if any, are likely to strengthen 
the link between testing and teaching? 

(5) What specific approaches to integrating testing and teaching have been 
developed, and how applicable are they to regular high school classrooms? 

These questions are addressed in the review in a somewhat distinctive fashion. 
A "table*^ that summarizes research and development work pertaining to each question 
is provided. These tables represent the substantive core of the review, ^nd are refrrred 
to throughout the discussion. 

1. What types of tests and e^essment procedures do teachers 
typically use, and for what purposes? 

At both th<» elementary and secondi^ry school levels, teachers report using a wl^e 
range of tests and assessmv«?nt procedures, from standardized tests, to district developed 
tests, to tests included^in curriculum materials, to self-constructec^ tests, quizzes and 
assignments, to classroom observations (Burry et al., 1982). Teachers use tests 
administered by a school district most frequently for planning and grouping decisions 
at the beginning of the year, though results from these tests also appear to play some 
part in grouping decisions made during the year and decisions 'bout students* report 
card grades (Burry et al., 1982). Classroom observation, and in some cases teacher- 
made tests, are used at the beginning of the year for grouping decisions, for verifying 
test information, and for developing an initial awareness of students* learning 
characteristics and attitudes. Observations and teacher-made tests clearly play a m-jch 
more central role in teachers' daily and weekly planning, and in grading, than do 
standardized or district-developed tests. Classroom . observations also help guide the 
moment-lo-moment, or "in flight," decisions that teachers make while interacting with 
students. 

Patterns of test usage are quite similar at the elementary and- secondary school 
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levels, with two notable exceptions. Secondary teachers use tests included in curriculum 
materials less frequently than do elementary teachers. For example, 68 per cent of 
the high school teachers surveyed in research done by Burry and associates (1982) 
indicated that results from tests accompanying curriculum materials were important in 
determining students* report card grades, whereas 93 per cent of the elementary school 

• 

a 

teachers surveyed found these tests important in grading. Secondary teachers aUo^use 
standardized tests less frequently than do elementary teachers, though the extent of 
standardized test use In secondary schools varies considerably by teacher subject matter 
specialization (Burry et al., 1982). 

More detailed summaries of teachers' ' use of standardized tests and classroom 
observations are provided in Tables 1 and 2, respectively. Detailed summaries have 
been prepared in reference to these sources of student information because they have 
been studied much more extensively than other aspects of classroom testing, such as 
the use of teacher-made tests. 

2, What pre teachers* attitudes toward arid concerns 
about testing? 

' Most research on teachers' attitudes and concerns about testing has focused on 
standardized and state or district-developed tests. By and large teachers appear to 
see some value in these tests, so long as they are used in conjunction with other sources 
of information on students. 

More specific research findings on teacher's attitudes to\vard and concerns about 
testing are presented in Table 3. It is noteworthy that teachers' attitude vary according 
to the level of knowledge they have about tests and the level of sc-iooling at which 
they teach, i.e., elementary or secondary. For example, consistent^^with the finding 
reported earlier that high school teachers use tests included in curriculum materials 
less frequently than do elementary school teachers is the finding that elementary school 
teachers perceive commercial tests to be of higher quality than do secondary school 
teachers. 43 



TABtE 1 

USES TEACHER3 MAKE OF STANDARDIZED TESTS 

j .' 

(1) l|n a recent large-scale study ^^of. test use In Americaa schools (Burry et al., 1982), the 
ijnajority of elementary schoqi teachers and a sizable number of high school teachers reported 
that they regarded standardized tests as one of many sources of information important for 
planning instruction and irouping students at the beginning of the year, and for adjusting 
instructional groups, during the year. Somewhat surprisingly, over ' 15 percent of the 
'lementary school teachers reported they also used standardized test results to help 
etermine students* report card grades. At the high school level, 12 percent of the English 
eachers and> 8 percent of the math teachers surveyed also used standardized test results 
or grading. 




(2) / Teachers appear to make very limited use of standardized test results in setting the pace 
/ of instruction (Barr, 1975). * 

(3) It is uncUar whether teachers view standardized tests as providing new and distinctive 
information, or whether such results are used primarily, to confirm judgments that they 
have already made. Interviews with a sample of 68 elementary school teachers from both 
urban and suburban schools (Salmon-Cox, 1981) indicate that standardized test results are .. 
mainly used to confirm teacher judgments, and not to help develop them. However, the 
importance attached to standardized test results by teachers surveyed ^in the study by 
Burry and associates cited above is not entirely consistent with this conclusion. 

(4) At leaist^me teachers use results from standardized tests to correct negative misperceptions 
they may\have developed about a student's ability. When test scores are lower, than 
expected these teachers ten(f to discount the scores. When scores are^ higher than expected, 
this is an indication to the teachers that they have overlooked something (Salmon-Cox, 
1981). Contrary to past claims, there is little evidence that test scorescause teachers • 
to iQwer expectations ior students. 

(5) At the elementary school level, teachers appear to assign about the same importance tM 
standardized test results in reading as in mathematics. At the high school level, Englisn^ 
teachers appear to place considerably more importance on stariardized tests than teachers 
of mathematits (Burry et al., 1982). 

(6) Teachers whose students generally perform poorly on standardized tests are less likely to 
use test results than teachers whose students generally do well on such tests (Yeh, 1978)* 

(7) Teachers with more teaching experience are more likely to use standardized test results 
than are less experienced teachers (Yeh, 1978). 
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TABLE 2 
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TEACHER OBSERVATIONS 

(1) The informal, minute-to-minute observations that teachers make in the classroori. should be 
>H distinguished clearly f"om the planned obserjwrtions of ' students' performance that 

\ chara|t4rlze "performance tests." Teachers use both kinds of observations, but informal 

observations are much more likely to focus on student involvement in learmjj^<^activities 
and the smoothness of classroom processes than on students' progress toward attaining 
desired learning goals (Clark & Peterson, 1981)^ ^ 

One study indicates that teachers rely heavily on their own observations 4nd students* 
classroom work to determine students' report card grides (Burry, at al., 1982). Another 
^tudy indicates "that information obtained through observation does?, not play a large role 
mUeachers' grading (Siiggins oc Bridgeford, 19iB2). The discrepancy between these findings 
may be based onthtfvaried meanings teachers ascribe to the term "observation." Perhaps 
formal observation of student performance and formal evaluation of class exercises figure 
orominently in grading, whereas informal observations - do not. This interpretation is 
consistent with evidence from the study by Cl^rk and Peterson (1981) referred to above. 

(^^ With respect to both formal and informal observations, teachers channel information into 
at least three different categories: (1) information that calls for an immediate response, 
e Sally begins to hit John on the head; (2) information that warrants delayed action 
and" continued dbservation, e.g., Bob is "off task" for a few moments, but may resume work 
on his own shorth^; and (3) information that needs to be stored for subsequent analysis, 
o 0 seveval stud%ts are having a hard time asking higher-order questions. Teachers often 
^lign the .^me^nd of information to different categories e.g., one teacher may respond 
' immediateljtto correct a student error, while another may continue to observe the student s 
response to see if an underlying error .pattern emerges (Joyce, 1981). 

(4) Though teachers rely a great deal on classroom observation to "fine tune" instruction, i.e., 
increase the time allotted to a particular activity, shift into a new activity, ask more 
questions about a topic or provide-more examples of a concep^, they rarely use afsessnjent 
information as a basis for changing their general approach to teaching, ^.g-. using je^^^^^^^^^^^ 
oriented vs. inquiry-oriented styles or a whole-group vs, smaU group management structure 

(Joyce, 1981). 

(5) Teachers use a range of procedures for evaluating information obtained through planned 
observation, the most common of which seems to be checklists (Stiggins & Bridgeford, 1982). 

(6) Teachers rarely ask another teacher to observe student performance or /ate a student 
product, but peer and student self-ratings are widely used (Stiggins dc Bridgeford, 1982). 

('7\ ronsiderable attention has been paid to the 'role that observation plays in teachers' 
Cessment anfinur^^^ of students' reading performance. It is evidently common 

?or eachers who have not received special training to observe inappropriate and irrelevant 
aspects of children's reading behaviors as they attempt to diagnose the causes of students 
reading difficulties (Polln, 1981).* 

rfl^ There is some evidence that teachers' 'observations of students' interpersonal classroom 
Lhavior are less accurate and stable thon their perceptions of students' overaU achievement 

- . . . . /fl^-v-* JL. Uaftere 1Q79A 



level in a subject area (Elmore & Beggs, 1972). 
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TABLE 3 

TEACHER ArnTOUES TOWARD AND CONCERNS ABOUT TESTING 

(1) Teachers generfiUy feel that testing motivates their students to study harder (Hurry et al..* 
1982). . " * 

(2) Teachers by and large have a more positive attitude toward standardized tests than 
measurement specialists attribute to them (Stetz k Beck, 1982). 

(3) Teachers generally feel that the direction of their districts' testing program are, to some 
extent, influenced by parental concerns about test results (Yeh, 1978). 

(4) The more knowledge teachers have aboui testing, the more likely they are to regard tests"* 
as useful (Yeh, 1978). 

(5) Teachers commonly assume that others outside the classroom, specifically parents, attach 
greater significance to standardized test results than they do (Salmon-Cox, 1981). 

(6) Teachers generally feel that there is a close match between what they are teaching and 
. the content of tests required by the state or district (Hurry et al., 1982). Nonethless, 

teachers often express Interest in "diagnostic" testsi that not only are closely matched to 
the curriculum with which they are working but available to them on a day-by-day basis 
(Salmon-Cox, 1981). 

(7) Teachers generally feel that tests developed in their district are very good (Hurry et al., 
1982). 

(8) Teachers generally feel that tests of minimum competency /proficiency /functional literacy 
should be required of all students for promotion to certain grade levels, or for high school 
graduation (Hurry et al;, 1982). 

« 

(9) Elemeqtary school teachers are more likely to pterceive tests of minimum competency as 
-unfair to particular students than are secondary school teachers (Hurry et al., 1982). 

(10) Elementary school teacher*' generally perceive commercial tests to be of higher quality 
then do secondary sfehool teachers (Hurry et il., 1982). ' 
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3. What do teachers generaUv know about testing, and what aspects of 
testlitg, or test usage, would they like to know more about? 

Teachers knowledge • about measurement Is limited.^ Measurement rarely is a 

significant component of ^eacher preparation programs. When inservice training in, 

* ' ... 
tesUng Is provided by a district it typic&Uy focuses on preparing for and administering 

district-required tests; little attention is given to the design or selection of classroom 
tests, or. the use of test Information. Secondary school teachers have received even 
'less training in testing timn elementary school teachers. Fortunately, there is evidence 
that teachers can and do benefit from .inservice programs linking testing and instruction, 
at least programs that are built around realistic classroom situations and that include 
ample opportunity f o; teachers to exchange Ideas' with their colleagues. A summary of 
research on teacher's knowledge about and trainln*^,in testing, and their expressed need 
for additional training, is presented in Table 4. 

4. What advances 'in the technology of testing, if any, are Kkely r . 
to^engthen the link between testing a nd teaching?. 

Unfortunately, few of the re-ent technological developmenU in testing have had, 
or are likely to have in the ne^r future, any impact on the testing. practices of classroom 
teachers. There is little reason to believe that teachers will use the technical procedures 
for test item specification and production that psychometricians have developed. Nor 
do complex ^proaches to test review and validation stand a chance of finding their 
way into the professional repertoire of the typical teacher. Some of the efforts to 
build tests* and test item pools that .relate closely to what teachers are teaching may 
extend the resources for testing available to teachers. But whether these efforts wiU 
affect teachers' testing practices ta any significant degree is an open question. 

' Table 5 summarizes key developments in the technology ofjtesting^li, also 
provides^ an overall assessment of the implications of thes6 developments for teachers 
and^aching. 
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TABLE 4 - 

TEACHERS* KNOWLEDGE ABOUT AND TRAINING IN ASSESSMENT, . \ 
AND THEIR EXPRESSED NEEDS FOR ADDITIONAL TRAINING \ 

(1) Teachers know littiS about basic measurement concepts (Rudman et><i^., 1980). 

(2) Based on interviews with a small sample of teachers it would appear that most teachers 
either have never had a course in measurement or have had only, one course. Fewer high 

, school teachers have taken courses in measurement than teachers in middle or elementary 
schools (Stigglns & Bridgeford, 1982). 

m 

' (3) The majority of elementary school teachers surveyed in a large-scale study (Burry et al., 
, 1982) indicated that their district had provided inservice\|raining in a number of areas 

related to testing, including procedures for administering tests, required by the state or 
district, and for interpreting information coming from these tests; procedures other than 
tests to assess student achievement; and procedures for relating instruction more closely 
to 'the content covered on required tests. However, relatively few teachers reported 
receiving training in test construction or selection, or in using test results^;^ instructional' ' 
improvement. . ' 

(4) High school teachers evidently have received considerably less training or assistanpre from 
their districts in test-related matters than elementary school teachers. The onearea in 
which a large number of high school teachers have received assistance is the administration 
and interpretation of ^'mandated" tests CBurry et al., .1982). 

(5) Without specific training few teachers design evaluatiori procedures that are tied closely 
to instructional goals- (Popham St. Baker, i?70). Nor do they show particular skill in 
identifying the causes of students' low performance, at least in the area of reading (Polin, 
1981). However, training in assessment and diagnosis docs improve teachers* skills in tlfese 
areas (Popham & Baker, 1970; Polin, 1981). 

• 

(6) A particularly promising approach to helping teachers use test information has been developed 
by Wanous and Mehrens (1981). This approach engages teachers in an analysis of d series 
of hypothetical, but realistic classroom situations that call for an instructional decision. 
Teachers are provided with a wide range of assessment information that is relevant to the 
decision under consideration. Exemplary ways of using th^s information to guide the decision 
are offered, against which teachers compare their own decision-making approach. Volunteer 
teachers received training using this approach in their own schools. The training provided 
exte/isive opportunity for teachers to work with colleagues on the problems posed in the 
hypothetical cases. B;- anJ large, teachers reported that the training helped them to design 
thei. own assessment p?ocedures; to select more carefully curriculum packages tha'^ included 
assessment measure.s: and to use data more systematically in making instructional decisions 
(Wanous & Mehrens, 1981). 

(7) Teachers at the elementary and junior high levels have expressed a need for-more knowledge _ 
about mea*'jring the affective and social dimensions of their classrooms. These teachers 
also hav;^ indicated an interest in extending their knowledge of assessment procedures in ^ 
areas other than the familiar ones of mathematics and reading (Wanous <5c Mehrens, 1981). 
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TABLE 5 

^ ADVANCES IN »THE TECIIMOLOGY OF TESTING AND THEIR 
IMPLICATIONS FOR TEACHERS AND TEACHING 

(1) New procedures have been developed for designing tests that correspond closely to the 
scope and emphasis of instruction, as reflected in the instructional materials vteachers are 
. using (Hanson & Colleagues. 1980). These procedures have been used effectively by the 
Southwest Reeional Educational Laboratory to develop program evaluation systems for local 
school districts. Information on students* learning in reference to a district's instructional 
«cope and sequence generaUy is collected In September* and at mid year, and used by 
administrators and teachers to Identify priorities for Instructional improvement. 

(2) Steps have been taken to develop assessment procedures for teachers in areas that 
traditionally have not been assessed systematically, e.g.. the resources for ^^essing speaking 
skills made available through the Speech Communication Association (Annadale, yirgima): 
the orocedures for evaluating student writing samples described in publications of the 
Northwesf Ro^ional Educational Laboratory (Spandel & Stiggins 1980) and the National 
Council of Teachers of EngUsh (Najmy. 1980); and tests like the "deductive logic and 
assum%on recognition" teste prepared by the Instructional Objectiv^ Exchange (Los 
AnS? These assessment resources ate intended for teachers' use. But the extent to 
• which teachers know about or use these resources has yet to be determined. 

r-ii Procedures also have been developed for creating test item pools that are linked e^^plicitly 
(3) implemented in a district (Valley Education Consortium 1982).., 

These poXpermit tealhers to generate tests based on what they are teaching at a level 
of sDecificity appropriate for a particular purpose, e.g.. pretesting, progress-checking. 
nosttesUnrend-^^^^^^^^ testing. The popls .also can be used by adminisfrators. to form 
tests sulta^^^^^^^^ monitoring and evaluation. Evidence Is not yet avai^ble. however 

' on v^e im^^^^^^ item pools on teachers' testlng-teachmg practices. 

tA\ Prnr*Pdures for assuring quality in the construction of criterion-referenced tests and for 
Ina yz?ng te^^^^^ been refined and extended In recent years (ferk. 19 0; Hamb^^^^^^^^^ 

?q8t^ Nitke 1974). However, these procedures, such/as those involved m reUability and 
con^uuct validity inve^^^^ or in using standard formulae for determining appropriate ' 
?est lenAhs^^d^^ be used by teachers at present. There is no reason to be^^leve 

that t^rsU"^^^^^^^^ in the' near future, unless time and resources avaUable to 

teachers for tes\ design, scoring, and Interpretation are dramatlcaUy increased. 

NPw nrocedures have been developed -for defining the domain of outcomes test is to 
^ ^ ^rLu^re ThSe i^^^^^^ Objectives." "Item forms." and "test ^specifications." 

Sowever' traS progra^ns have not been designed to teach teachers to use these Procedures. 
Thrchtef deielope^r of practitioner-oriented domaln-specificatibn strategics. James Pop^^^^^^ 
rfeDorts that "he\is unable to reduce the process to a form^ that Is directly teachable 
fponhlm 1980 D ^17) " Even if teachers were trained In domalh-speclflcation strategies. It 
Is n'ot cieL tharthey would have tLe to use them on a regular basis In their actual work. 

(n A number of highly technical procedures for .generating test Items from a domain have 
^ ^ ipn d^vGl^^^^^^^^ and llnguisiic transformations (Millman. 1980; Rold & 

S^na 1^^^^^^^^^^ T^eseTroce^^^^ v^hlch co>monly e e used with the aid of computer 
uchnol^D^. ap^^^ limited. If any. utility i'or the classroom teacher. • ^ 

. (7^ Some progress has been made^n assessing the underlying <^<»g"itive strategies students^^^^^^ 
^ L "arryTouT various learning tasks In specific subject areas <Adams Col to. 1979 
Ber ^terV 1979). Procedures for assessing cognitive strategies, and for identifying 
?nonnrriiate oJ suboptimal uses of a strategy.- are In an early stage of development. 
S?^^^^^^^ l^ind. however, have been incorporated with apparen success 

fn seT/ral d^agnostically-oriented Instructional programs >.ost notably in Project Torque 
ischwartz & Taylor. 1979). wnich focuses on ♦he teaching of. computation and measurement 
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5. What specific approaehes to integrating testing andneaching have been 
developed, and how applicable are they to regular high school classrooms? 

Av^'ariety of specific approaches have been developed for integrating testing and 

instruction^ most notably mastery learning and computer assisted instruction. Less 

commonly use^ approaches include adaptive testing, answer-until-correct testing methods, 

self -scoring andl self-diagnosis procedures, and tests that simulate real-life problem 

situations. Brief descriptions of each of these approaches are presented^ Table 6. 

V ' ■ ■ 

Specialized approaches to integrating testing and teaching probably will not be 
used in typical classrooms vuithout the assistance of computer t^hnology. These 

\ ^ ■ 

approaches call for a higl^ level of individualized instruction. Uuder present conditions, 

particularly in high^schools, this level of individualization cannot be managed effectively.\ 

, ' « . 

Even mastery learning, which was developed independently of advances in micro-computer 
technology, poses serious classroom management problems for most teachers, for it adds 
a ^eat deal of paperwork to their teaching load. Teachers will need computer assistance 
to manage the large quantity and refined quality of inforngation on individual student 
achievemfent that most^models for integrating testing and teaching call for. 

Ck)nclusion 

A good deal of research has been done over the last decade that relates to the 
issue of integrating testing and teaching. For the most part this research is encouraging. 
It suggests that teachers find value in many types of tests, and use test information for 
a variety of purposes. It also suggests that teachers are aware of the limitations of 
tests, and use them selectively in combination with other sources of information on 
students. In addition, research indicates that teachers can increase their effectiveness 
in designing and using tests through inservice training programs. 

Research and development efforts also suggest a note of caution. Scientific 
progress made in test design, scoring, and validation does not automatically translate into ^ 
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TABLE 6 

SPECIFIC^ APPHO ACHES tO INTEGRATING TESTING AND TEACHLNG 

AHfintive TestlnP This Is en approach to testing in which the particular items administered to 
e^^minee al"^^ ^^ in time depend upon the examinee's performance on items administered 

o rfr^uious time The-^ number of items administered, may vary. e.g.. examinees who are clearly 
above or b^ orsome stand^^^ may be assessed with relatively few items, -^he difficulty level 
aooye °r pvaminees who are doing very well on one set of items are 

• tUn!stSaTerS.'-|on>puTr^^^^^^^^^^ used to ffcilital adapUve te^lngCMiUman. X980). 

. «.n««r-Untn-Correet (AUG) Testing Methods. These are testing proceduresWfuch the student 
f«^»rLued to Sue se ectine from item alternatives until the correct alternative ^ chosen. 
'Aul%"^ocfdures°eS the teacher to review a student's entire respo ttern and thereby 
gain a better understanding of the causes of low perfprmance (Wiloox. 1982). 

r-nmnuter-Assisted Instru ction (CAD. In most CAI programs, testing is included in the form of 
Ee^Lts Degress tests, and mastery tests; The computer determines what each student sl^ould 
So next on the basis ut test results: .e.g.. go on to more advanced material, receive addiUonal 
fnVructidn or practice, etc. (If remedial activities prescribed by the computer do not result m 
L^tl^f^Mo^v oeXmanie t^ the computer tells the student to see the teacher). Some . 

if rLf mrtTlearnine te^e abiUty levels, learning style, e.g, auditory or 

^^.^nUive s vie e I abstract ^^S^^ot the learner (Charp. 1979). At present 
C« anTs'pSrar; ^^^^^^^^^^^ to teach the ^^^f 

CAI profraras currently are being designed to develop more sophisticated skills (Martin. 1981). 

- • -A c=i/ nioano<!i<! Procedures. These are procedures in which students score tests 
are omvird with anS ^^ booklet that offers explanations of why each 
themselves, and are proviaea °" '"J^V,,. Alternatives The College Board's Career Skills 

keyed answer is «°"s f^^^ P-^^/^^J^^tu L. 1^^^^^^^^^ sheets accompanying the 

Assessment program for example^^^^^^^^^^ ^ ^^^^^ the layered answer 

program are ^"'f ^d so ^i^^^ score their own answers on one part; the 

^sheets can be separate''. Student, ^^st booklet gives a rationale for each 

"arerTpd ^p^e^enUTuratir ff hor^^^^^^^^^^ on the test can be appUed in practice. 

■ ^ , ,h.t Ti^nch Some^ests are so closely connected to learning.'that they represent a form 
^^Trysn^^Vests oresented t^^ pilot trainees by fUght simulators, for example, are excellent 
of teaching. The tests presentea lo P ' j emergencies, respond quickly and 

examples o tests t^f teach The trainees l^^^^^^ llediately (KrumboUz. «82). Simulators 
roTherieLrn^prob^^^^^^^^^^^^ Career Decision Simulation constructed 

by Krumboltz, Hamel, and Scherba (Krumboltz, 1982). 

tn moctoru ipflrnino the curriculum is clearly sequenced so thai foundation 
Ma stery Learni ng, In mastery learning ^2?,,, Advanced content is introduced. Students are 
^^;3i5tn;Hrii^ are ^^^if^^;^^^^^^^ in the curriculum they should 

assessed prior to instruction to ^ete mine precisely at jvnai^^ ^^^^ ^^^^^ 

begin work. Instructional me is free to va^^^^^^^ 

students learn. Similarly. ^ f fo monitor student progress toward attaining 

in -student learning ^Jy^^VrtpLuied fn the cur iculum, and to identify needs for instructional 
the learning ^"^f ,,^3^,^"'^^^^^^ of an instructional unit. These tests 

adjustment. Mastery J-^^^ts .^^7" new work or whether instructional "recycling" i needed. 

VttsnU^iraVepI^l^rf^^ 

I9^^^rrK^n=| XTy%rarte?;™Srapp"roach to teaching hi^h school 
chemistry (Swanson & Denton. 1976). 
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benefits for the classroom teacher. Nor do technically sophisticated approaches to 



\ 



integrating testing and teaching have direct Utility for the typical teacher. Research- 
based procedures for strengthening the link J^etvveen teaching and tesL^ng \^11 have to 

^ ] 
be carefully adapted for use in the nation's schools and classrooms* 
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APPENDIX 3 



A Questionnaire Used to Assess High School 
Teachers* Testing Practices 



A Que!:tionnaire Used to Assess High 
Seh:>ol Teachers* Testing Praetiees 



The questionnaire presented on the pages that follow was designed to help us gain 
a better understanding of high school teachers* approach to teisting and the use of test 
informat».^n. It^also was intended to yield information on twfe-^st-related resources 
available to teachers and the attitudes and concerns teachers have about testing. 

The questionnaire was completed in the spring of 1983 by 523 teachers in 9 high 
schools in Oregon and 6 schools outside the state. The schools in the sample consisted 
of some of the schools in which members of the Teacher Advisory Panel taught and 
some^ whicA were involved in a goal-based education project sponsored by the Northwest 
Regional educational laboratory. All full-time teachers in the sample schools were 
asked to complete the survey. Approximately 64 percent did so. 

Results from the survey were tabulated in May of 1983. The principal of each 
school in the sample received a summary of results for his or her building that same 
month. A summary of responses to each item on the questionnaire is available upon 
request from the Teaching Research Division of the Oregon State System of Higher 
Education. ^, 

Unfortunately, the information produced by the survey provioed only limited help 
in developing the Handbook. This was largely because the results simply confirmed that 
teachers use a variety of test item formats, place major emphasis on the use of tests 
for grading, generally have limited formal training in assessment, and so forth. In 
restrospect, it might have been better to give the questionnaire to teachers who are 
known to integrate instruction and assessment in a thorough way and those who are 
known to make only weak connections between instruction and assessment. This may 
have helped to enhance our understanding of the practices and attitudes associated with 
different approaches to teaching and testing. 

The survey did, however, provide a basis for developing the teacher survey 
instrument that has bee^ used in the CEPM research project described earlier in this 
report. More information about the refined version of the teacher survey and its use 
will be provided in subsequent reports to the NIB, made in connection with the CEPM 
project, V 
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TEACHING KESRARCli DIVISIOK . 
OREGON STATE SYSTEM OF HIGHER EDUCATION 

A Survey of High School Teachers* 
Tiling Practices, and Attitudes Toward Testings 

Background 

This survey asks you abouv your approach to student testing. It also a^ks about 
your use of test results. For purposes of the survey, testing is considered as any 
systematic way of gathering information on students. 

Findings from the survey wiU be included in a handbook for high school teachers 
on linking testing and teaching. The handbook is being developed by Teaching Research, 
^state-supported agency specializing in educational research ^and development, through 
ontiact with the National Institute of Education. It is intended that the handbook 
w^U help »»igh school teachers make better use of tests as aids in instructional planning 
and decision-making. A copy of the handbook will be sent to your principal's office 
when it is completed, in the spring of 1984. The handbook will contain a summary of 
results found through the survey. A preliminary report of results wiU be sent to you 
toward the end of this school year. 

Directions 

Please answer the questions . candidly. Information is needed on what you do 
- not on what you feel you should do. It is recognized that no teacher can do all that he 
or she would like to do. Also, do not put your name on the survey; responses are to 
be anonymous. 



Thanks for your vviUingness to complete the survey. 



Testing Formats and Procedures 

Which of the follnwlng tests an(^ assessment procedures do you generally use? 
(Clunk M\ Ih.il «)|)ply') 

final exams In-class exercises 

mid-terms projects and reports 

unit tests homework a&slgnments 
quizzes other (please specify) 



2. 



Please indicate the frequency with which you use each of the following 

test formats (check the appropriate space). 

/ Frequency of Use 

i Very Occasion- 



Format 



• multiple choice 

• true/false 

• matching 



« fin in the blank or problem- 
solving 

• short-answer essay' 

• extended essay 

« other student products, e.g., 
a painting, musical composi- 
tion, mechanical drawing * 

• student performance, e.g., 

answering questions out loud; 
typing a letter; dribbling a 
basketbal I 

•open book tests, I.e., testB In 
which students can use their 
notes or books 

take home tests, i.e., tests that 
students do at home 



Often 



Often 



ally. 



Sel dom 



ERIC 



58 



•-2- 



PUase Indicate the frequency with which you use additional sources 
of information about student learning or student characteristics. 

t Frequency of Use 

Very Occaslon- 

Source of Information Often Of ten ally Sejdom 

• previous. teachers' records, ' 

reports 'or grades 

/ 

• information provided by a . 

: student's parents 

• standardized tests of 

achievement 

• tests of student aptitudes . — - 

• state or district developed ^ 

lests 

• measures of student atti- • 

tudes or Interests ^ 

• measures of student learn- 

Ing styles 

• other (please specify) . ^ 



provide a review of the material to be covered on the test 

give a practice test 
^ construct the test .sing questions that have already been 
discussed in class 

provide training irv test-tal<ing skills 

other (please specify) — — 
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5. Which of the following testing practices do you use on a regular basis 
when working with how-ability students? (Chectc all ttiat apply.) 

. * ' 

tell the students not to worry If they don't do very well on a 
test 

v.; 

give the students special tfests matched to their ability level 

give the students "^xtra help so they have a better chance of 

doing well on tests 

give the students less advanced worlc and correspondingly easier 

tests < « 



other (please specify) 



6. How much class time do your students spend in a typical week taking 
tests ^and reviewing test results? (Check the appropriate space.) 



0-30 minutes 
* 

30 - 60 minutes 
60 - 90 minutes 
over 90 minutes 



/ Analyzing, Managing and Reporting Test information 

7. When analyzing results from objective tests, I generally; (check the 
appropriate space) 

determine each student's overajl score on the test, but do not 

perform any kind of 1 tQ|m analysts; I.e.^ I rarely look at how 
students do on individual test items 



analyze responses to a few i te^is In which I am particularly 
interested 

analyze responses to each test I tern 

I don't give objective tests 
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Whl^h of the following resources for scorfng and managing test results. 
If any, are aveHfibie to you: (check aU that apply) 

-paid adult oc student aides 

^volunteer adults or students 

scoring machine 

in my classroom 

In the building x. 
can use for any objective test I give 
can use only for some exams (e.g., final). 



In another bul idlng 

I 

can use for any objective testVUglve 

can use only for some exams (e.g., flna!) 



other (please explain) 



9 How much time do you typically spend each week outside of class scoring 

tests or grading papers of projects? (Please estimate to nearest 15 minutes.) 



0 Listed below are three types of measures of student learning. About how 
lonq after students complete each. of these measures do you generally let 
h^ know how they did on It? (Check the appropriate spaces. If you do not 
reTularly use one of the nfeasures listed, leave the spaces next to it blank.) 



Within Within Within Within 

2k Hours 2-3 Days a Week 2 Weeks 
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Quizzes — 

Unit Tests 

Mid-Terms 

When discussing a student's academic progress with his/her parents, 
which of the following practices, If any, do you usually follow? 
(Check all that apply.) 

tell the parents the student's scores on various tests that 
you have given 

show the parents examples of the answers that the student has 
written to your test questions 

indicate how the student's scores on your tests compa'-e to the 
' class average * 

telll parent*, how .the student has performed on general aptitude 



f 
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Us Inq Test Information 



12. what ways do you use test Information? (Check all that apply.) 

lo ^roup sti^dents for Instruction, or particular learning 
act I vU lei 
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to decide whether to teich the class ^uiore about a topic or to 
go on to another topic 

• * ■ 

to decide whether a particular student knows enough that he or 
\he can skip a unit 

to decide whether a partlefilar student needs extra help 

to decide what type of teaching approach to Implement, e.g., 
lecture vs. discussion; sma,11 grou^vsi whole class activity, 
«c. ■ _ , ■ • J , 

_ to assign grades to students 

^ to evaluate the effectiveness of a course 

other (please specify) . 



13. Which of the following practices do you use on a regular basis when 
a large percentage of your class does poorly on a test? (Check all 
that apply.) 

reteach the class the material on which the test was based 

_ reteach only those students vho did poorly 
allow students,^ retake the test, or a comparable test 



ask students who did well on the test to provldi some form of 
in-claus help or tutoring to those who did poorly 

move on to the next unit anyway 

other (please specify) : 
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\k. Listed below are a number of factors that might explain why a large , 
percentage of a class does poorly on a test you have prepared. Place 
a check on the line that Indicates your perception of the frequency 
with which each of these factors comes Into play. 

Frequently a Sometimes Seldom 

a cause of a cause of a cause of 

poor performance poor performance poor performance 

, • 

TiME: Not enough time was 
available to teach the 
concepts and skills thoroughly, 
or to provide students with 
sufficient opportunity ^to 

practice or apply them. . -t— 

FEEDBACK PROCEDURES: Too 
little information was ♦ 

provided to students on .- .. . 

their progress toward ' 
attaining the learnl g 

outcomes that were covered , : ' 

on the test. — : 

hETHODS/MATERlALS: The • ' 

instructional methods and 

materials used were not 

well suited to the students. 

or to the type of learning • 

outcomes expected. . ' 

CONTENT: The concepts and 
skills that students were 
expected to learn were too 
complex, too abs^act, or 

too many in number. — > — ; . ■— " 

TEST DESIGN: The test did 

not match closely what was , 
taught, or required students 

to derronstrate what they ^ 
had learned in unfamiliar 

ways. — ^ 




Whith of the following practices do you use on a regular basts when 
a student consistently does poorty on your tests? (Check all that 
apply.) 

talk with the student ' 

^ talk with other teachers about the student 

make a point to watch the student during class periods ^ 

give the student extra help 



give the student ^ diagnostic test to se^ if he or she has the 
prerequisite skills needed to do the regular coursework 

give the student less advanced work 

talk to the student's counselor or parents 

explore the possibility of the student's transferring to a less*^ 
advanced course 

. • ' 'if 



V 

64 



-8- 



16 Listed below are ■ number of factors that might explain why an 

INDIVIDUAL STUDENT consistently docs poorly on tests you have pre- 
pared. Place a check on the' line that Indicates your perception o1^. 
the frequency with which each of these factors comes Into play. 



Frequently a 
a cause of 
poor performance 



Sometimes 
a cause of 
poor performance 



Seldom 
a cause of 
poor performance 



The student was frequently 
absent or tardy during the 
, time, the concepts or skills 
*coA^red on the test were 
taught. 

the student did not partici- 
pate fully or consistently In 
learning activities, or was 
distracted from learning by 
disruptions In tTie classroom 
environment. 

The student djd not complete 
homework assignments. 

The student lacks the 
prerequisite concepts or 
skills needed to succeed 
In the course* 

The student has difficulty 
learning jeoncepts or skills 
through ythe type of Instruc- 
tional Method or materials used 
with thi class as a whole, or- 
within we time constraints 
normal I y imposed. 

The student' has no interest; 
in the topics being taught. 

The student has difficulty 
demonstrating what he/she 
knows or can do In the form 
called for on the test, quiz, 
or exercise. 

The student was 111 or unduly 
distracted the day the test 
was given. 

The student needed more time 
to finish the test. 
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17* Which of the following practices do you use on a regular basls^when 
a student consistently scores at the top of the. class on your tests? 
(Check all that apply.) 

talk with the student 



talk with other teachers about .the student 
give the student more advanced work 
_ ask the student to tutor other students In th« class 
give the student more opportunity to work independently 
talk to the student's counselor or parents 



explore the possibility of the student's transferring to a more 
advanced class 

other (please specify) ^ 



% L 

\ 




8. Do you use tests that accompany textbooks on a regular basis? 

Yes No 

If yes, which of the* following uses do you generally make of these 
tests? (Check all that apply.) 

to supplement Information from your own tests 
____ to use instead "of yo^r own test* 

to provide guidelines for designing your own tests 

other (please specify) . — 
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19 Listed below are a number of factors that teachers consider when 
assigning grades. Enter In front of each factor the weight It 
carries in most of your courses. Make the percentages add up to 100. 
If you do not use a source that is listed, leave the space in front 
of it blank. 1 



'% final exam 
'% mid-term 

% unit tests « 

\ projects or reports 
% homework assignments 
% quizzes or in-class exercises 
% performance on a standardized test 
% performance on a district-administered test 

J% class participation 
% effort made by the student to learn 
% other (please specify) 
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20. Do your testing practices, Including your use of test Information, differ 
from one course to another? 



Yes No 
If yes, please explain what you do differently In each course, and why* 



} 
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Concerns About Testing 

Some teachers have concerns about the use of tests in thel'' classrooms. 
Others don't. Check the concerns listed below that you have at this 
time. 

Students often seem uninterested In learning anything that they 
are not going to be tested on or assigned a grade for. 

The scoring and analysis of tests Is quite time consuming. 

Students who do f-oorly on tests often feel they arc stupid and 
have little Incentive for studying hard. 

Many valuable aspects of learning dc not seem to be measurable 
'through tests, e.g., creativity, wisdom, fairness. 

Students put too much emphasis on how their test scores compare 
with others In the class. 

Too much, Instructional time Is consumed by tests requi red by 
~^ the stat4 or federal government, or by the district. 

other (please specify) „. — — 



Some teachers have concerns about the use of tests in education 
generally Others don't. Check the concerns listed below that 
you have at this time. 

The public puts too much emphasis on national tests, such as 

' the SAT, that aren't good measures of what is being taught in 

individual courses. 

Tests of student achievement are sometimes viewed as a means 

for monitoring teacher performance without taking into account 

the different ability levels and backgrounds of the students 
teachers have to deal with. 

The money spent on testing would be put to better use If it 
""'were used to purchase I nstruc" ional materials. 

other (please specify) .. 
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Interest In Learni ng More About Test-Related Issues and Practices 

23. How much Interest do you have In attending a one-day workshop on 
dcs gninq iind using tests as an aid to Instructional planninq and 
dcci si on-making? ^ 



No Interest Some interest onslderable Great Interest 

Interest 



2A. Regardless of how you answer the prtivlous question, please rank the 
following possible workshop topics In order of Interest to you. 
Place a 1 In front of the topic of most interest, a 2 In front of 
the topic of next Interest, and so on. 

design tests that are matched closely to what I have 

taught 

— ^ how to deslgR^jests In areas of learning that are generally 
considered td be hard ts measure, ^..g., creativity, wisdom, 
leadership 

to use tes^ results as a guide for planning remedial or 

enrichment activities 

^ow to sc^re and manage test Information with the aid of a 

micro- computer 

• to report test results to students, parents, or administrators 

other (please specify) 
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background Information 

25. What subject do you teach? (Check the appropriate space.) 

Career. Education Math 

English Physical Education 

Fine Arts ^Science 

Foreign Language Social Studies 

Health Vocational Education 
Home Economics Other (please specify) 

26. How many years have you been teaching? 



27. What professional degree do you hold? 

BA or BS 

BA or BS + 30 hours 

Master's Degree 

Master's + 30 hours 

Ph.D. or Ed.D. 



20. How many courses in test design and measurement have you taken In 
preparing to be a teacher? 



When were these courses taken? 



25. Have you participated in any workshops or other inservlce training in 
testing or use of test information? 



Yes No 



If yes, please describe the workshops or training sessions and indicate 
when you participated in them. 



30. Piease write the names of your school and district. 

District- 
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INTEGRATING 
TEACHING AND TESTING 



For Further Information 
about the program and 
related materials ~ 
write 



THE OREGON EDUCATIONAL 
COOPERATIVE PRESS 
Todd Hall 
Western Oregon State College 
Monmouth, Oregon 
97361 



or call 

Area Code 503-838-1220 
Ext. 391, 392 or 393 



T^ac 



The HamSKMk For Integrating Teaching 
and Testing in the High School, and 
related staff development materials, were 
developed by Drs. Glen Fielding and Del 
Schalock of the Teaching Research 
Division, Oregon State System of Hi^^her 
Education. These resources for staff 
development build on a long history of 
work TrT^regon on the integration of 
teaching aid testing, and the use of test 
information in program management at all 
levels of schooling. It also builds on a 
long historytof work in Oregon on school- 
based staff development, and on the work 
of the Valley Education Consortium in 
both of these areas. Both the Handbook 
and the Staff Development Guides were 
pilot tested in Oregon and Washington 
schools. 



A Staff Development Program 
For Improving the Quality of Instruction 
In High Schools 



Developed under contract with 
The National Institute of Education 
1984 
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A STAFF DBVBLOPMENT PROGR 
FOR TEACHERS AND 
ADMINISTRATORS 



Created to help local districts take 
advantage of recent advances in instruc- 
tion, testing and the use of test 
Information 

. in aiding student learning; 
. in guiding instruction; 

in developing a curriculum and 
preparing plans for a course; 

. in evaluating and improving instruc- 
lional programs^ 

provides the understanding and skills 
needed for departme.its to effectively 
implement instructional programs that 
depend heavily on testing and the use of 
test information, for example 

, mastery learning; 

. goal-based, or outcome-based 
instruction; 

^ . individualized instruction. 

Emphasizes the ererging concept of 
curriculum alignment in testing, and builds 
on the established technology of criterion 
referenced testing. 



A LOW-COST PROGRAM 
THAT CAN BE IMPLfeMENTED 
WITH EXISTING PERSONNEL 

Designed for use by a faculty of a 
department, but may be used by more than, 
one department at a time. 

Operated by the school's Principal, a 
Department Chair, a "Lead Teacher," and 
a resource person in or outside the 
district who is Itnowledgeable about 
recent developments in testing. 

The basic training program requires only 
two days of released time for teachers. 
Three additional days can be devoted to 
advanced training, but thesQ are optional. 

The basic training program gives teachers 
the understanding and skills they need to 
start where students are, monitor learning 
gains that are made, and plan or adapt 
instruction accordingly. 

The basic program takes as its focus one 
or more "units" of instruction currently 
being taught. Instructional units are 
prepared during the training program that 
fully integrate teaching, testing and the 
use of test information. 

The advanced training program focuses on 
an ent're course, and on the connection 
between courses and programs of 
instruction. ' 



A COMPLETE SET OF 
PROGRAM RELATED RESOURCES 



The staff development program makes use 
of 

. a Handbook for integrating teaching 
and testing in high school programs; 

. a Resource Guide for Teachers that 
is designed to be used by members 
of 8 department as a team; and 

. a Pfogram Planning Guide for 
Administrators and Leaf! Teachers 

that is designed for Superintendents 
and other central office administra- 
tors responsible for initiating the 
program, and Principals, Lead 
Teachers and Department Chairs who 
are responsible for implementing the 
program. 

In combination these documents make up 
an integrated, self-contained training 
system that can be used by local districts 
to accommodate local needs and 
resources. 
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V 

An Outline of the Evolution and Completion of Contract #400-82-0013: 
Developing a Teachers* Handbook for Integrating Teaching 
and Testing in High School 



4/15/85 



Products Called for 
in Original Contract 

1. Literature 
Review 



2. Survey of high 
school teachers' 
testing practices 



Teachers' 
Handbook 



Exemplary 

teaching-testirig 

materials 



Modifications Made and Approved 
During the Course of the Project 

None 



Originally it was thought thai 
information from the survey 
would be used as a guide in 
designing the Handbook. As it 
turned out, the survey results 
had limited utility in preparing 
the Handbook, but the survey 
has proved very useful in a 
related research study 
sponsored by the NIE. (See 
notes in the right-hand column.) 



The H andbook went through a 
more extensive process of 
review and revision then originally 
planned (see pages 4-10 of the 
Final Report). 

Originally we were to Work with 
teachers in developing a separate set 
'f illustrative teaching-testing proce- 
dures. We did work with teachers in 
designing such procedures, but rather 
lhan treating them as separate docu- 
ments we have integrated them in the 
Handbook and in the Resource Guide 
for Teachers. The HandtK)ok and 
Resource Guide contain numerous 
examples of different types of learning 
goals, goal-based instructional 
activities and assessment procedures, 
and ways of using test information 
in the instructional process. 



Date Submitted 
to the NIH 

. January, 198^ 

. Additional copies were included in 
the Final Heport, submitted 
February, 1985. 

. The final version of the survey was 
submitted in Sprii.^, 1983. 

. Information on the use and 
refinement of the survey is 
containea in the-Final Report. 
Related information has been . 
provided in progress reports 
subtuiCted to the NIE in 'jnnection 
with a project on high school test- 
ing sponsored by CEPM through 
Grant NIE-G-81-"01iO. 

The final version of the Handbook 
was submitted in February, 1985. 



Included in the Teachers' Handbook 
and the Res ource Guide for Teachers. 
(The final version of the Resource 
Guide accompanies *his outline and 
is described below.) 
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Products Called for 
in Original Contract 

5* A description of 
how i\)fi Teachers^ 
Handbook could 
be implemented 
in school-based 
inservice 
programs. 



6. A dministrator's 
Guide 



Modifications Made and Approved 
During the Course of the Project 

It wab ^car from the outset of the 
project that we^ needed to provide 
guidelines on using the Teachers^ 
Handbook in schctol-based staff develop- 
ment programs. However, it was nqt 
until the second phase rf the project 
that we recognized the need for a 
comprehensive, step-by-step imple- 
mentation manual for teachers. While 
the Teachers' Handbook serves as a 
mini-text, the Resource Guide for 
Teachers that was developed through 
the no-cost extension of the project 
providps sequenced learning activities. 
The Resource Guide represents a more 
elaborate end thoroughgoing response 
to the issue of implementation than 
we originally anticipated It also goes 
beyond a focus oh individual classrooms 
and contains guidelines for integrating 
teaching: and testing at the program- 
level. 

The original contract called for the 
development of an administrator's 
guide, which would help district and 
building administrators plan and support 
local inservice training programs based 
on the Teachers' Handbook . As the 
project unfolded it became clear^/ 
however, that administrators wou^d 
need to work hand-in-hand with "lead" 
or "master" teachers in structuring and 
carrying out the training, and in sustain- 
ing its effects. We pilot tested an 
inservice program in whinh lead teachers, 
with support from their principals, 
facilitated inservice workshops in their 
buildings. Based on the success of the 
pilot test, we decided to expand our 
original notion of an administrator's 
guide into a Planning Guide for Lead 
Teachers and Administrators . This 
Planning Guide does all that the original 
administrator's guide was to do, but also 
identifies major roles and responsibilities 
for lead teachers. 



Date Submitted 
to the NIE 

The field test version of the 
Resource Guide for Teachers 
accompanies this outline. The 
Resource Guide will be further 
tested and refined during the 
remaining months of the CEPM/NIE 
grant NIE-G-81-0110, and submitted 
to the NIE in final form at the 
completion of that grant. 



A first draft, "field review" version 
of the Pl anning; Guide for 
Administrators and Lead Teachers 
accompanies this outline. It will be 
more fully developed and pilot tested 
during the remaining months of the 
CEPM/NIE grant NIE-G-81-0110, and 
submitted to the NIE in final form at 
the completion of that grant. 



7. Final Report This was submitted in February, 

1985. 
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